Canary: AI Clinical Change Detection
Shipped the first AI use case at the company and built a foundation for building AI features on a 20 year old established EHR.
Problem
The company was making its first serious investment in bringing AI into the core clinical product. The AI research team had already been fine-tuning an LLM to summarize patient progress notes for nurses, benchmarking and getting clinicians to label data for the use case. Adoption was low at ~1% of eligible users. Iterations to improve the model was not improving usage.
Approach
I joined as the first PM to operationalize our AI product development.
Get to the real problem under the surface. The research team started with summarizing daily progress notes as it's the natural first application with LLMs. I started by front loading discovery calls with my physician advisory board (that I assembled from building PW), and learned the core insight: the most valuable and easily missed signal for practitioners is the "small changes" that are not obvious and don't trigger any alert on its own but compounds over time to lead to a preventable hospitalization.
One nurse lead gave it the perfect analogy, "it's like canary in the coal mine", the early signals that can make a big difference in patient's outcomes if intervened early. That's way more useful than daily summaries of progress notes.
Optimize for how the feature embeds in user's workflow. From analytics, I realized the feature's placement was suboptimal: it was buried inside a specific page that nurses had to navigate to, and the note types it summarized represented only about 10% of a nurse's daily documentation.
These central insights led to the feature being named "Canary", and drove immense clarity for the research and AI platform teams to revamp it.
What I built
Redefined the JTBD: from summary to early change detection. I reframed the AI task as change detection: surfacing variances from baseline, new symptoms, trending patterns of decline or improvement, and updates to medications, diet, or care plans that are often ignored. This required new data labeling, new annotation guidelines built around what clinicians consider a clinically meaningful "change," and retraining the model with that definition. This gave the research team a more clinically grounded objective to optimize for instead of basic summarization quality.
Created a globally accessible AI panel. The original summarization feature was in the progress notes page because of its positioning as "note summarization". But nurses and physicians navigate across many tabs in the patient chart, not spending all their time in the "note" page. I designed a floating AI panel with a globally accessible (yet not annoying) entry point across any page in the patient chart. Whether a nurse was passing meds, doing a shift handoff, preparing for a clinical standup, or writing a progress note, the change summary was accessible and embedded in the notification model. This reframed the feature to behave like a clinical companion.
Defined what "change" precisely means. Without a rigorous definition of "change" (which can mean anything from a fall to a subtle dietary shift), the model either overwhelms users with noise or misses the important signals. I worked with clinicians to map two tiers of change: one is change from baseline, two was variance trends: patterns of decline that individually get missed but collectively signal something worth attention. This taxonomy became the labeling framework for the AI research team, and made the model's output feel truly useful to the clinicians.
Impact
| Adoption | From ~1% of eligible users to ~20% usage in early access, with customers requesting access proactively |
| Use case coverage | From summarizing ~10% of a nurse's daily workflow to surfacing changes across the full scope of clinical documentation |
| Clinical value | Clinicians reported catching changes they would have missed, particularly when covering unfamiliar patients |
| Model direction | Redefined the AI research team's objective from general summarization to clinically-grounded change detection |
| Product pattern | Established the global AI panel as a reusable pattern for future AI features across the product, rather than embedding each feature in a single page |
| Organizational shift | Proved the value of PM-led use case discovery for AI products, setting a precedent for how the company approached subsequent AI features |
Learnings
The kernel PM skills are even more valuable in building with AI. The creative act of asking the right questions to get to the crux of the real problem still proved to be the most important foundation to building, AI or not. During this time (2024), the foundation models were not where they are today, so starting with the deep and focused understanding of the customer's real pain, the one that they couldn't even articulate well, was the major differentiation to applying LLMs in a meaningful way. (Today (2026), there's a real (additional) case with starting from the model to test the ceiling of what it can do.)
In regulated, high-stakes domains, don't treat users as early adopters. The clinicians on the floors weren't actively looking for "AI features to try". In fact, it's more likely that they have inherent doubts about their usefulness and potential disruption to the daily flows they've honed over the years. It takes a zero friction and zero disruption integration into their existing workflows to win their trust, ie. don't ask them to do something drastically different from their existing workflow. The global panel worked because it was never intrusive, but proved to be useful in a surprising way. For AI products in healthcare and similar domains, "embedded and familiar" beat "novel and impressive."