LLMs for Clinical Documentation
A Wearable Device Supporting Reflective Clinical Documentation Using Large Language Models
Role
UX Researcher & AI Consultant
Timeline
April 02 - June 04
Team
3
Introduction
This project, developed during my Master's thesis, explores how Large Language Models (LLMs) can augment clinical journaling in physiotherapy without compromising practitioner-patient interaction or reflective practice. Through interviews, fieldwork, prototyping, and testing, I investigated how AI can automate tedious aspects of documentation while supporting professional reflection and maintaining practitioner agency.
Context
Physiotherapists face time constraints that hinder comprehensive journaling, leading to reduced patient interaction and missed opportunities for reflection. This project emerged from fieldwork, identifying that tools should reduce screen time and cognitive load while preserving the reflective practices vital to clinical care.
Discovery Phase
I conducted literature reviews, expert interviews, and contextual inquiries with physiotherapists to understand their workflows and journaling practices. This revealed tensions between efficiency, patient presence, and reflective practice, emphasizing the need for tools that adapt to physiotherapists' tacit knowledge rather than enforcing rigid schemas.
Ideation & Prototyping
Using Brainwriting, Crazy 8s, and material exploration, I generated and iteratively refined concepts, focusing on the affordances and limitations of LLMs as a design material. Sketching and prototyping explored interaction models, form factors, and metaphors, balancing automation with practitioner agency.
Storyboarding
Storyboards generated with LLMs based on sketches and narratives allowed envisioning practitioner-system interactions while surfacing opportunities for system agency adjustments, context awareness, and the preservation of professional judgment in workflows.
Material Exploration
Prototypes explored configurations such as treatment-transcript journals, treatment-plan generators, and proactive dictation, testing both GPT-4o (cloud) and Phi-3 (local) with varying temperature settings to understand hallucination risks, tone-of-voice tuning, and reliability in clinical contexts.
Wizard of Oz Testing
Using a Wizard of Oz setup, physiotherapists engaged with the prototype in realistic treatment scenarios, reviewing AI-generated journals post-session. This revealed the need for clear explainability cues, output chunking to reduce cognitive load, and the system’s ability to adapt to the physiotherapist’s tone.
Final Prototype
The final prototype is a wearable device that captures audio and video during treatment, processes data locally, and generates structured journals for practitioner review. Capacitive touch sensors allow practitioners to control data capture intuitively, maintaining agency and patient presence.
AI-Native UI & Reflection
The system produces drafts that practitioners can edit and review, supporting reflective practice and enhancing clinical decision-making. A feedback loop allows the system to learn the practitioner’s style, gradually shifting from assistant to co-pilot, aligning with practitioner workflows and trust-building patterns.
Explainability & Trust
Explicit XAI cues highlight uncertain outputs, empowering practitioners to assess and verify AI-generated content. This transparency supports trust and aligns with the professional standards of healthcare documentation.
Metaphors of Agency
Metaphors such as 'Assistant,' 'Intern,' 'Co-pilot,' and 'Agent' were used to conceptualize the system’s evolving role, adjusting its level of autonomy based on trust, practitioner comfort, and contextual needs while maintaining practitioner control.
Ethics & Privacy
The system processes data locally, upholding GDPR standards, with clear visual cues for recording states to maintain transparency. Testing used mock data, ensuring privacy while exploring realistic workflows, and emphasizing practitioner autonomy.
Automation & Augmentation
The system embraces Shneiderman’s 'Goldilocks zone' of thoughtful automation, augmenting clinical workflows without replacing practitioner judgment. It reduces administrative burdens while preserving opportunities for reflective practice, enhancing the quality of care.
Reflection
Empirical testing showed that the prototype reduced documentation time while preserving patient interaction and practitioner reflection. Iterative refinement revealed the importance of explainability, tone adaptability, and non-verbal cue integration for trust and usability. Future work includes integrating computer vision for movement analysis and refining prompt tuning for context awareness. This project contributes to interaction design research by demonstrating how AI can be ethically and transparently integrated into clinical workflows to support, not supplant, practitioner expertise.
Written in collaboration with ChatGPT-4.5