Measuring and developing healthcare skills in new ways has never been more timely for two reasons: 1) ongoing workforce shifts and 2) emergence of new tools. Workforce composition, especially in primary care, is undergoing a transformation in both business and practice models. As a result, there is a lot more variability in training and experience of clinical talent in the labor supply. This phenomenon is true across other specialties too, but professional silos do not help us build a more capable workforce. More importantly, we have to recognize that we can’t advance primary care if we don’t tap into all of its resources appropriately. This raises an essential question of what does good care look like? If it wasn’t for the current technology renaissance, we would not be able to tackle this at scale.
Figuring out how to best leverage healthcare workers across all levels of training is hard. That’s why it’s helpful to learn from labor transformation efforts in other industries. The team at Anthropic is looking at how specific tasks and associated skills will be impacted through automation or augmentation rather than trying to conceptualize how entire occupations will change (Figure 1). Large companies have been created to measure “skills intelligence.” Some notable examples include Workera (general purpose skills intelligence) and DX (developer productivity). Similarly, incumbent healthcare workforce development platforms like HealthStream and Amplifier service health system stakeholders but have yet to achieve meaningful reach within the complex realities of primary care.
Evaluating clinician performance, especially in chronic care, is hard for multiple reasons. For example, what is considered an appropriate referral for new onset atrial fibrillation in a middle-aged woman will depend on the local context (e.g. private practice vs. hospital affiliations) and the payment model of the practice (e.g. FFS vs. capitation). Additionally, condition specific skills and competencies (e.g. hypertension management) have long feedback loops that are difficult to surface in fragmented episodes of care and have fuzzy boundaries of onset and resolution. Bringing visibility to clinician performance has been a passion of mine. This year, that vision has matured from an idea into a business with paying customers. Here’s what resonates with them:
Identifying the skills index of their organization and individual clinicians (Figure 2)
Upskilling each clinician based on alignment of their educational gaps with organizational goals (Figure 3)
Doing all of this without additional administrative friction while meeting regulatory requirements for chart reviews and continuing education
Scaling business operations on a foundation of clinical excellence
We are often asked (particularly by non-clinicians): why not intervene with performance insights at the point of care? First, this is incredibly hard to do well. Deploying tools that dictate behavior in real time is fraught with unintended consequences like automation bias and de-skilling. With the advent of autopilot, the aviation industry has left us with decades of learnings about the impact of technology on high-stakes human performance. Second, the space between the clinician and their patient is already congested with continuous interruptions like care gap alerts, best practice advisory (BPA) notifications, and proliferation of other clinical decision support (CDS) widgets. Third, CDS AI will likely face regulation first. As their rapid, and sometimes cavalier, adoption accelerates, patient harm stories are inevitable - and staying in the game will demand serious capital.
With AI making its way into clinical workflows, understanding how to evaluate clinician performance will get even messier. We’re already seeing early studies on how models like AMIE are outperforming PCPs across a multitude of domains, including empathy and medical decision making. Paradoxically, some evidence suggests that AI tools perform better in isolation than with human involvement. This signals that our ability to wield them is fairly primitive. To change this phenomenon, a new set of AI competencies are emerging for integration into medical education. Through this work, it will be more apparent why humans are not the weakest link in healthcare and that many tech startups are automating the wrong part of the care experience.
Healthcare has fallen behind other industries in developing meaningful insights into workforce performance. Over $100M has been invested in chart review companies in recent years, only to see their technical prowess converge on compliance, revenue cycle management, and monetizing work for pharma. We’re overdue for a different way of leveraging technology to make a difference in healthcare operations. For this reason, we regularly convene a clinical skills intelligence (CSI) working group for healthcare leaders (e.g. Chief Medical Officers, Quality Officers, Medical Directors). Similar to how experts in aviation safety come together to share best practices, our sessions focus on problem solving around operationalizing good care. We trade notes on:
Quality improvement initiatives (how to identify educational gaps, verify competencies, and develop new clinical protocols)
Clinical team management (candidate vetting and retention strategies and personalizing upskilling interventions)
Evaluating AI technology (decision frameworks for implementing pilots, governance and legal implications)
Three principles guide our work: 1) everyone deserves good care; 2) good care comes from quality clinicians; and 3) every clinician can improve. Developing a more capable workforce is a tall challenge and as Larry Bauer puts it “stop trying to change the system alone.” We want to meet others committed to empowering good care. Encourage your clinical leaders to apply for CSI membership (free to XPC customers).
P.S. If you are an engineer, and would like to work on the problems we’re tackling, reach out to paulius@xprimarycare.com
P.P.S. We hope to see you at our upcoming virtual IdeaFest on Nov 1. It’s going to be an interactive 4-hour unconference to collectively tackle the question: what does good care look like? across all healthcare stakeholders. You’ll get to network with many brilliant minds shaping care delivery. There is no cost to participate, but space is limited. Sign-up here.
Thank you for sharing this inciteful summary.