1 of 15

UTAH DEPARTMENT OF COMMERCE

HEALTH AI PARTNERSHIP WEBINAR

The Utah Sandbox

Balancing Innovation & Regulation

JONATHAN OLSEN, DO

Physician · Utah Medical Licensing Board member

Speaking in an individual capacity

ZACH BOYD, PHD

Director, Utah Office of Artificial Intelligence Policy

Utah Department of Commerce

29 OCTOBER 2026

2 of 15

Disclosures & perspectives

Two speakers; distinct roles and institutional responsibilities.

UTAH DEPARTMENT OF COMMERCE

Sources: Webinar draft disclosures

Jonathan Olsen, DO

Utah Medical Licensing Board member; Medical Board AI Working Group participant.

FSMB AI workgroup participant; specialty-society AI committees.

Speaks individually—not for the Medical Board, FSMB, or other organizations.

Zach Boyd, PhD

Director, Utah Office of Artificial Intelligence Policy (OAIP), Department of Commerce.

Senior author of Evidentiary Expectations; FSMB AI workgroup participant.

Speaks officially; his views represent OAIP positions.

Discussion questions do not necessarily express settled agency policy.

AI disclosure: Claude Fable 5.1 (Anthropic) and ChatGPT (OpenAI) assisted with organization, formatting, and sourcing. The speakers reviewed the final material.

Corrected disclaimer: Jonathan speaks as an individual; Zach represents official OAIP positions.

ORIGINAL SLIDE 2

Disclosures

Jonathan Olsen, DO

Federation of State Medical Boards – Regulation of AI in the Practice of Medicine Workgroup Member; Nominating Committee Member

Utah Medical Licensing Board – Member; Medical Board AI Working Group

American Board of Allergy & Immunology – MOC Question Writing Committee Member

American Academy of Allergy, Asthma & Immunology – AI Committee Member

American College of Allergy, Asthma & Immunology – AI Committee Member; Practice Management Committee Member

Utah Medical Association – AI Committee Member

Zach Boyd, PhD

Director, Office of Artificial Intelligence Policy, Utah Department of Commerce

Senior author, Evidentiary Expectations for Healthcare AI Sandbox (August 2026)

Federation of State Medical Boards – Regulation of AI in the Practice of Medicine Workgroup Member

Views expressed are our own and do not represent official positions of the Utah Medical Licensing Board, the Federation of State Medical Boards, or the State of Utah.

AI Disclosure: AI (Claude Fable 5.1) was used to organize, format, and source our ideas but all content has been carefully reviewed and is our own.

3 of 15

Innovation vs. regulation

Shared objective: enable useful clinical AI without dispensing with patient protections.

UTAH DEPARTMENT OF COMMERCE

Sources: Original webinar; OAIP framework

Innovation

Autonomous AI may expand clinical access, including for underserved Utahns.

Graduated autonomy can be supported by rigorous continuous quality assurance.

Sandbox evidence can inform more durable regulation.

Regulation

Autonomous clinical AI still has limited evidence in many settings.

Professional accountability systems were built around human clinicians.

Medical boards and clinicians should help shape safeguards.

The question is what evidence and oversight each clinical use case needs.

4 of 15

Utah’s sandbox: a short timeline

From the 2024 law to supervised clinical AI deployments in 2026.

UTAH DEPARTMENT OF COMMERCE

Sources: Utah Commerce; OAIP FAQ; webinar draft

2024

SB 149 establishes OAIP and its AI Learning Laboratory, including regulatory mitigation agreements.

2025–26

Doctronic tests renewals of existing chronic-disease medications, using phased physician review.

Spring 2026

Medical Licensing Board raises concerns and begins closer consultation with OAIP.

August 2026

Board AI Working Group engages with OAIP; Evidentiary Expectations published.

October 2026

New pilots, health-system agreements, and an open network of evaluators are announced.

ORIGINAL SLIDE 4

Utah's Timeline

Two years from statute to AI-issued first-time prescriptions.

2024 | SB 149 creates the Office of AI Policy (OAIP) and the AI Learning Laboratory. Regulatory mitigation agreements (RMAs), co-signed with the regulating agency, can temporarily adjust how state licensing and scope-of-practice rules apply.

Oct 2025 – Jan 2026 | Doctronic pilot: AI renewal of 190 chronic-disease medications. Physician review of the first 250 renewals per drug class, then retrospective review, then 5–10% spot checks.

Apr 2026 | Medical Licensing Board writes a letter of concern OAIP responds that Phase 1 includes physician review of every renewal.

Aug 2026 | The Board's AI Working Group begins meeting regularly with OAIP. OAIP publishes Evidentiary Expectations for Healthcare AI Sandbox (Schwarze and Boyd)

Oct 5, 2026 | OAIP Press Conference: Nolla Health (first-time acne topical prescriptions), August AI (Rx renewals), Expect Fitness (pelvic-floor PT); umbrella agreements with University of Utah Health and Intermountain Health; six third-party evaluators including CHAI and Stanford CERC.

Source: Utah Commerce, Utah OAIP FAQ, Becker's, STAT

5 of 15

Which clinical AI are we discussing?

The key distinction is who makes the clinical decision.

UTAH DEPARTMENT OF COMMERCE

Sources: FDA discussion paper; Utah Medical Practice Act

ASSISTIVE

Documentation and retrieval

Clinician decides

AUGMENTATIVE

Decision support and differentials

Clinician decides

AUTONOMOUS

System issues a diagnosis, plan, or prescription

System decides

“Agentic” means multi-step action or tool use; it does not inherently imply autonomy.

Diagnosis and prescribing can be medical practice under Utah law; agreements adjust specified state requirements.

ORIGINAL SLIDE 5

Agentic AI Models: Differing Degrees of Involvement

Assistive — ambient documentation, literature retrieval: clinician decides

Augmentative — decision support, differential generation (e.g., OpenEvidence): clinician decides

Autonomous — system issues the diagnosis, plan, or prescription, a human sees it later, or not at all: clinician does not decide

“Agentic” describes capability (multi-step action, tool use). It can exist at any of the three levels above.

FDA's proposed two-axis frame (Aug 2026 discussion paper): how independently a function acts × severity of a wrong output.

Everything that follows is about the third row only.

What can a model do with or without human supervision?

Source: FDA discussion paper

ORIGINAL SLIDE 6

When Does Software Practice Medicine?

Diagnosing and prescribing are the practice of medicine under Utah law [Utah Medical Practice Act, Utah Code Section 58-67-102].

A regulatory mitigation agreement is a promise not to enforce specified licensing, professional-conduct, and telehealth provisions against a named company, for a defined scope, for a limited term.

If a waiver was necessary, the activity is the practice of medicine. Then: who sets its standard of care, and who is disciplined when that standard is breached?

FSMB leadership's position from 2024: boards regulate licensees, not tools; responsibility should scale with the level of autonomy and human oversight.

Utah’s sandbox removes licensing requirements to allow algorithms to operate in clinical spaces.

Source: Sheppard Mullin, STAT (FSMB leadership)

6 of 15

What the sandbox is designed to test

A temporary, evidence-generating approach—not state approval of a product.

UTAH DEPARTMENT OF COMMERCE

Sources: OAIP, Evidentiary Expectations §§ 2, 5–7

Quality

Care comparable to an appropriate clinical reference.

Pre-specified review criteria.

Safety

Monitor for bodily and psychological harm.

Escalate, correct, or regress when risks emerge.

Public benefit

Demonstrable value for Utah patients.

Meaningful access and equity considerations.

Moderate entry barriers + high expectations for ongoing quality assurance

Initial pilots last up to 12 months, with limited extensions; agreements can differ from the default posture.

ORIGINAL SLIDE 7

Defining the Sandbox

Two aims: identify AI use cases that are safe and beneficial; identify regulatory frameworks that can govern them (1).

Three hypotheses every pilot must be able to test: non-inferior quality of care; bodily and psychological safety; public-health benefit in Utah (2.1).

Stated posture: “low market entry barriers and high expectations for continuous quality assurance” (2.2).

Status of the document itself: a “default negotiation posture”; individual agreements “may deviate”; it “should not be interpreted as establishing legally enforceable requirements” (Summary).

Term limits: a first test period cannot exceed 12 months, extendable twice; then ordinary rules apply in full.

“Being accepted into the sandbox is not an endorsement or approval of a company or its product by the State of Utah.” — OAIP

Source: Evidentiary Expectations, Utah OAIP sandbox overview

7 of 15

Who governs autonomous clinical AI?

Oversight is distributed across existing institutions—not a single new regulator.

UTAH DEPARTMENT OF COMMERCE

Sources: Agency roles; webinar governance discussion

OAIP

Sandbox agreements, evidence expectations, and policy learning.

DOPL / Medical Board

Clinician licensure, practice standards, and discipline.

Utah DHHS

Traditional health-facility and public-health oversight.

Consumer Protection (DCP)

Traditional authority over vendor conduct and representations.

FDA / federal bodies

Device regulation and other federal roles where applicable.

Systems & evaluators

Deployment accountability, independent evidence, and clinical oversight.

Which responsibilities belong with clinicians, institutions, vendors, and government?

Added the traditional facility oversight role of Utah DHHS and the vendor oversight role of Utah Division of Consumer Protection.

ORIGINAL SLIDE 9

Who Should Be Regulating Autonomous Clinical AI?

State medical boards (FSMB leadership): govern within existing professional accountability structures, boards do not license machines.

FDA (Aug 2026 discussion paper): competency-based premarket evaluation “inspired at a high level by how physicians are trained and evaluated”.

Academics (Bergman, Wachter, Emanuel; JAMA 2026): a new federal Office of Clinical AI Oversight; USMLE-level competency, supervised deployment “analogous to residency,” defined scope, biennial renewal, federal preemption.

Utah: a state AI office plus DOPL, with Board working-group input and third-party evaluators, under one- to three-year contracts.

Everyone agrees someone should, but it isn’t clear who should

Source: STAT (FSMB), FDA, STAT (Bergman), Utah Commerce

8 of 15

Seemingly low risk labels can hide high-risk situations

The activity performed and the consequences of error both matter.

UTAH DEPARTMENT OF COMMERCE

Sources: OAIP pilot descriptions; FDA framework

Doctronic: renewals

Renews an existing prescription written by a clinician.

No dose changes or controlled substances in the pilot described.

Begins with structured physician review.

Nolla: full patient acne journey

Uses photographs to assess acne for an initial clinical decision.

The primary risk is misidentification of non-acne conditions.

Initially requires two independent Utah-licensed physicians to approve.

A low-risk medication does not necessarily mean a low-risk clinical decision.

9 of 15

Evidence before deployment

What evidence is enough to start a highly supervised pilot?

UTAH DEPARTMENT OF COMMERCE

Sources: OAIP, Evidentiary Expectations § 5.2

Potential evidence

Experience from other jurisdictions.

Historical back-testing and internal A/B tests.

Simulated encounters and credible predictive models.

In some cases, applicant-generated evidence at entry.

Why the threshold is not obvious

Evidence can become stale as models and populations change.

Broad systems are difficult to exhaustively test.

AI models consistently overfit to predeployment data, giving an overly optimistic impression.

External validation is important as pilots progress.

Regardless of predeployment evidence level, early deployment monitoring must be strong.

ORIGINAL SLIDE 10

Pre-Deployment Evidence

Acceptable evidence: performance from other states or countries, back-testing on historical cases, company-internal A/B tests, simulated encounters, a credible world model (5.2.2).

“OAIP can consider and may accept proposals with all pre-deployment evidence generated by the sandbox applicant without external validation” (5.2.3). External validation is expected by the year-2 renewal.

OAIP's rationale: pre-deployment evidence “can become stale” from population, behavioral, and model drift; broad-scope AI is “hard if not impossible” to test fully in an RCT (2.2.1).

Medical Board: reasonable for a Phase 1 with licensed physician review of every case, less clear when fully autonomous with physician retrospective review.

“Moderate pre-deployment evidence” and “a credible argument that the product is safe”

Source: Evidentiary Expectations

10 of 15

How a pilot earns more autonomy

Three stages, with decision rules established before progression.

UTAH DEPARTMENT OF COMMERCE

Sources: OAIP, Evidentiary Expectations § 6.1 and Appendix C

Silent trial

Use an independent clinician reference before any clinical action is taken.

→

Full retrospective review

Review every system decision retrospectively.

→

Sample review

Review samples after pre-specified stage gates are met.

Gates: agreement, sensitivity / specificity, and severe-error bounds.

Revert after a plausible systematic harmful incident or a failed performance threshold.

Original slide provides numeric margins, confidence bounds, and O’Brien–Fleming alpha-spending details for speaker reference.

ORIGINAL SLIDE 11

Phased Deployment

Three mandatory stages for autonomous systems: silent trial with independent clinician reference → retrospective review of every decision → sample-based review (6.1.1).

Pre-specified statistical rule: agreement, sensitivity, and specificity must each clear a floor set at clinician–clinician agreement minus a margin (agreement ≤5 points; sensitivity ≤2 points where a false negative is dangerous) (6.1.2).

Exact binomial critical values on an O'Brien-Fleming alpha-spending schedule: a tool performing at the floor advances with ≤2.5% probability; the upper confidence bound on severe misalignment must be <2% (Appendix C).

Regression is mandatory after a harmful incident with a plausible systematic cause, or when a gated measure falls below its floor (6.1.3).

The stage gates are more statistically rigorous than most clinical credentialing.

Source: Evidentiary Expectations

11 of 15

Who reviews—and against what standard?

A statistical gate is only as good as its reviewer and reference standard.

UTAH DEPARTMENT OF COMMERCE

Sources: OAIP Evidentiary Expectations §§ 4, 6

The reviewers

Should have licenses in scope and relevant specialty training.

In current pilots, some reviewers work for the developer.

Automation bias and independence should be monitored.

The benchmark

Clinician agreement is an early proxy, not patient outcomes.

Outcome measurement and external validation strengthen evidence but require much stronger infrastructure.

Medical Board AI Working Group contributing to standards development.

The long-run objective is better patient outcomes—clinician judgment is our most proximal, available benchmark.

ORIGINAL SLIDE 12

Phased Deployment Review / Standard

Who reviews? Licensure within scope and specialty match are required (4.1.2), but in current pilots reviewers are employed or contracted by the developer. The framework itself names reviewer automation bias as a risk to monitor (2.2.2).

Against what standard? Clinician agreement, not patient outcomes. OAIP calls outcome measurement the robust approach and accepts agreement as “proximal” (4.1.1). Doctronic, first five months: physicians agreed with 91% of AI renewal recommendations and judged 69% of escalations appropriate. What was in the other 9%?

Who stands at the gate for the public? Since spring 2026, proposals are presented to the Medical Licensing Board's AI Working Group before approval.

A gate is only as good as the clinician reviewing.

Source: Evidentiary Expectations, TechTarget, Utah OAIP FAQ

12 of 15

The residency analogy: useful but difficult to operationalize

Graduated autonomy is the parallel; equivalence to a human trainee is not.

UTAH DEPARTMENT OF COMMERCE

Sources: OAIP framework; original webinar; JAMA

Where it helps

Supervision decreases as performance is demonstrated.

Independent action requires ongoing quality assurance and peer review.

Offers regulators familiar terms for staged deployment.

Where it breaks

Residents enter with standardized educational credentials.

Model errors may correlate differently from clinician error and can recur at scale.

A model version is not a licensee; its “attending” may be unclear.

Correction: clinician errors are not necessarily independent; model error patterns are different, not categorically more correlated.

ORIGINAL SLIDE 13

The Residency Analogy

Residency works because the resident, the attending, and the program are each accountable to someone. Which of those is the algorithm?

Where it holds

• Graduated autonomy: supervision steps down as performance is demonstrated.

• Continuous quality control after independence: M&M, peer review, liability, licensure review.

• Adopted explicitly by the framework (2.2.2), by Bressman (JAMA IM 2025), and by Bergman, Wachter, and Emanuel (JAMA 2026).

• It gives regulators a vocabulary they already trust.

Where it breaks

• A resident has already cleared a high bar (medical school, USMLE) before touching a patient. The sandbox standard isn’t proven (yet).

• A resident's errors are independent. A model's errors are correlated: one flaw, every patient, until the next monthly report.

• A resident is one licensee who can be remediated or removed. A model is a version. Which version was credentialed, and what happens when the base LLM changes?

• The attending is a licensee the board can reach. Who is the attending for an algorithm?

Source: STAT (Bergman), Penn LDI, Evidentiary Expectations

13 of 15

Accountability, consent, and equity

Traditional patient protections remain central inside a regulatory sandbox.

UTAH DEPARTMENT OF COMMERCE

Sources: OAIP Evidentiary Expectations §§ 2, 5–7

Accountability

Allocate responsibility and recourse along the care chain.

Insurance, incident notices, and patient-transition plans.

Clarify developer, health-system, and clinician roles.

Patient agency

Disclose AI involvement and temporary sandbox status.

Provide feedback channels and a meaningful opt-out.

Include patient perspectives in oversight.

Equity & access

Assess target groups and access barriers at proposal stage.

Analyze subgroup performance as needed at any stage.

As sample size increases, we are powered to look closer at subgroup performance.

Who is accountable when an autonomous clinical decision causes harm?

Correction: subgroup performance is considered during proposal review, revisited as needed during pilots, and has specified later reporting requirements.

ORIGINAL SLIDE 14

Accountability

Required of applicants: an accountability framework along the delivery chain and patient recourse comparable to non-AI care (5.1); specialty malpractice coverage; no liability disclaimers in terms of service; 24-hour incident notification; a funded termination and patient-transition protocol (6.2).

The open question: in Stage 3 (sample review), the only licensees in the chain are the developer's medical director and reviewing physicians. The Board's only lever is their licenses. Is that fair to them, and is it protective for patients?

Candidates for shared liability: authorizing state agency · developer · deploying health system · medical directors employed by the developer · physicians using the model in an adopting health system.

The framework asks applicants to allocate accountability “where harm can be prevented most effectively.”

Source: Evidentiary Expectations, Utah Commerce

ORIGINAL SLIDE 15

Patient Transparency and Consent

Required (2.3): disclosure of AI involvement; disclosure of sandbox status (“preliminary, temporary, and revocable”); a feedback channel to OAIP; a genuine opt-out unless infeasible. This exceeds most direct-to-consumer telehealth disclosure today.

Not required: a patient advocate in the review process; independent ethics review; any test of whether a subscription-app disclosure screen constitutes informed consent to an unvalidated care pathway.

Recommendation we share: a patient advocacy seat at the table in Utah which is still needed.

Is a sandbox pilot a clinical service or a clinical study? If it is a study, where is the IRB?

Source: Evidentiary Expectations

ORIGINAL SLIDE 17

Equity and Access

Appendix A, item 2: underserved populations are “less likely to adopt these solutions” because of distrust, infrastructure, and tech-literacy barriers; applicants must name the population they aim to reach and the barriers they expect.

Subgroup performance reporting, including vulnerable populations, is required by the end of year two (7).

The access case rests on underserved Utahns. The framework's own appendix says they are the least likely to adopt.

Source: Evidentiary Expectations

14 of 15

Other governance details

UTAH DEPARTMENT OF COMMERCE

Sources: OAIP framework; webinar synthesis

What the framework expects

Monthly reporting and rapid notices of severe harm.

Monitoring for drift, incidents, and changed systems.

Stage regression when important safety signals emerge.

More complete outcome surveillance as data systems mature.

What remains unsettled

Detecting harm when patients seek treatment elsewhere.

How strongly clinician agreement predicts outcomes.

Who decides when oversight is reduced or restored.

How to address version changes and correlated failures.

What evidence and oversight justify each increase in autonomy?

ORIGINAL SLIDE 16

Post-Deployment Monitoring

Commitments: monthly reports; 24-hour notification of severe harm; a defined incident list (missed diagnosis, allergy, contraindication, out-of-formulary prescribing); drift and change-management monitoring; third-party validation favored (5.3, 6.3, 6.4).

The dependency: OAIP's own concern list includes harms made “invisible for detection” when a patient leaves the AI care path and presents elsewhere (2.2.1). Infrastructure to track hospitalizations and deaths across the sandbox is something OAIP “aims to collaborate” on building; it does not yet exist (4.2).

“Most data will not be available until after deployment.”

Innovation

This is why continuous quality assurance, not a pre-deployment trial, is the right instrument.

Regulation

Until the outcome infrastructure exists, retrospective and sampled review means the patient population is the test set.

Source: Evidentiary Expectations

ORIGINAL SLIDE 18

Balancing Innovation & Regulation

What is clear | What is unclear

Innovation is outpacing regulation; abstention is not a strategy. | Whether a low pre-deployment bar is acceptable when the deployment population is the test set.

Assistive and augmentative AI should be adopted broadly. | Whether clinician-agreement rates are a sufficient reference standard absent outcome data.

Graduated autonomy with pre-specified gates beats a binary approve/deny. | Whether Stage 3 (sample review) is “supervised” in any sense a medical board would recognize.

Drift is real; continuous quality assurance is non-negotiable. | Whether the medical board should have a formal role at stage progression and year-2 renewal.

Patients must know they are in a sandbox and be able to opt out. | Whether the residency analogy survives the version-control and correlated-error problems.

Multiple stakeholders, including patient advocates, belong in the process. | Whether “safe and superior” is a near-term expectation or a hypothesis still untested.

ORIGINAL SLIDE 19

Remaining Questions

1 If an agreement must adjust licensing law to let an algorithm prescribe, is the algorithm practicing medicine? If yes, who sets its standard of care?

2 Is a sandbox pilot a service or a study? What would change if we called it a study?

3 Who is the attending physician for an autonomous model, and what happens to its credential when the base model changes?

4 Should subgroup performance and outcome-linked monitoring be entry gates rather than year-2 deliverables?

1 of 15

2 of 15

3 of 15

4 of 15

5 of 15

6 of 15

7 of 15

8 of 15

9 of 15

10 of 15

11 of 15

12 of 15

13 of 15

14 of 15

15 of 15