An Introduction to the Healthcare AI Lifecycle

The Healthcare AI Lifecycle

At the heart of modern healthcare is evidence-based medicine (EBM). EBM is a data-driven decision-making approach, utilizing data from clinical trials and observational studies to guide clinical decisions. This approach to guiding diagnostic and therapeutic decision-making requires synthesizing multiple loosely connected pieces of scientific literature and assessing whether a patient’s presentation aligns with findings or criteria from previously published studies. Consistently applying EBM decision-making in a busy clinical environment, especially one that caters to a wide range of patient conditions and acuity levels, can be daunting.  Additionally, the current EBM scientific study system leads to a long time between when data are collected and when findings are applied to clinical practice.

Enter artificial intelligence. Although artificial intelligence has become a loaded term with many different meanings, my working definition is relatively simple:

Artificial Intelligence (AI) is intelligence, perceiving, synthesizing, or inferring information demonstrated by machines (non-human/non-living entities).¹

We can use AI tools, also called models, to encode existing knowledge gathered from clinical trials or medical experts. Additionally, many of the recent advances in AI have been driven by a set of tools known as machine learning.

Machine learning (ML) techniques seek to build models that learn or improve performance on a task given more data.²

ML offers powerful techniques to create data-driven prediction models that align well with the objectives of EBM. ML methods can be used to rapidly develop models that learn the relationship between patient attributes (age, heart rate, etc.) and the patient’s future outcomes (e.g., risk of developing diabetes). By leveraging AI and ML, we can significantly improve our ability to predict outcomes, personalize treatments, and ultimately enhance patient care.

ML models are already in clinical use, aiding the synthesis of complex medical information. The Food and Drug Administration (FDA) has approved over 600 AI systems for various medical tasks, ranging from analyzing electrocardiograms to detecting breast cancer through mammograms. Beyond these certified applications, health systems and health information technology (HIT) players, like electronic medical record (EMR) vendors, are developing and deploying AI systems that don’t require FDA certification. These tools are designed to assist physicians by providing risk estimates. The goal is to integrate these tools into medical decision-making processes and enhance the precision and effectiveness of patient care.

The landscape of healthcare AI systems is incredibly diverse. AI tools can enhance or inform patient care, clinical decision-making, or operational efficiency. Despite their variety, these systems fundamentally operate the same way as information processing tools. So, whether used by patients, clinicians, or health systems, these tools share commonalities in their development and utilization. Ensuring the safety and effectiveness of these systems requires a standard series of steps, collectively referred to as the healthcare AI lifecycle. This lifecycle encompasses all the necessary phases to bring a healthcare AI system from conception to practical medical application.

Understanding the Healthcare AI Lifecycle

The healthcare AI lifecycle can be divided into two principal phases:

• Development, when AI tools are created
• Implementation, when AI tools are used in practice

It’s crucial to recognize that the journey of developing healthcare AI is continuous and doesn’t conclude once a model is deployed. This ongoing commitment is vital to the success of AI in healthcare. Like all other software development, healthcare AI requires refinement and iteration to adapt to new data, evolving medical knowledge, and changing clinical needs.

The distinction between development and implementation is somewhat artificial. Effective AI integration into healthcare systems necessitates a blend of these phases, mirroring the principles of software engineering best practices. Feedback loops between development and implementation ensure that AI tools remain relevant, accurate, and beneficial. This iterative process, akin to agile methodologies in software engineering, is vital for maintaining the safety, efficacy, and utility of AI systems in a dynamic healthcare environment.

We can foster a more holistic approach by aligning AI development and implementation with established software engineering best practices. This perspective encourages ongoing collaboration between developers and healthcare professionals, ensuring that AI tools evolve with medical advancements and real-world clinical experiences. Bridging these phases can enhance the robustness and reliability of healthcare AI systems, ultimately leading to better patient outcomes and more efficient clinical workflows.

Healthcare AI Development & Implementation Lifecycle. The development phase focuses on creating AI models, encompassing task selection, data access, data preparation, model training, and validation. The implementation phase involves integrating these models into clinical care, including technical integration, prospective validation, workflow integration, monitoring, and continuous updating. This lifecycle highlights the iterative and interconnected nature of AI development and implementation in healthcare, ensuring models remain effective and reliable in real-world clinical settings.

Overview of Development

Development encompasses the multifaceted processes involved in creating an AI model; it can be broken down into several key steps:

1. Task Selection: Model development should begin with a process where experts identify the specific task or clinical problem an AI model should aim to address. This step involves a collaborative effort between clinicians and data scientists, ensuring the model’s relevance and potential impact. Your expertise is crucial in this process.
2. Data Access: Securing necessary datasets is often a significant hurdle. Healthcare data, being highly sensitive and complex, requires careful handling and usually involves navigating regulatory and privacy constraints.
3. Data Preparation: Once data are obtained, they must be processed and transformed to be suitable for model development. This step demands a unique blend of clinical and technical expertise to handle the intricacies of healthcare data, including cleaning, normalizing, and annotating the data.
4. Model Training: Once data are prepared, the actual development of an AI model can begin. This involves selecting appropriate algorithms, training the model, and fine-tuning it to optimize performance. This step is iterative and often requires multiple adjustments to achieve desired performance characteristics (e.g., the targeted accuracy and reliability).
5. Validation: After training, the model must undergo rigorous evaluation, where we assess its performance in different ways. There are several related ways to do evaluation; these include internal validation within the development environment and external validation, which tries to replicate other real-world settings.³  The goal is to ensure the model’s effectiveness and reliability in clinical practice. External validation can be particularly challenging due to data-sharing restrictions, but it is essential for assessing the model’s real-world applicability.

Each of these steps is crucial in creating robust and reliable AI models. By methodically addressing task selection, data access, data preparation, model training, and validation, we can craft tools that significantly enhance medical decision-making and patient outcomes. Many of these steps may be viewed as the purview of developers and engineers; however, physicians should be actively involved in every part of the journey. Clinical expertise ensures that the AI tools are technically and clinically sound, yielding tools that are performant, practical, and aligned with patient care realities.

Overview of Implementation

Implementation involves integrating and utilizing an AI model within clinical care settings. Although the use of AI models in clinical care should only commence once a model has been thoroughly validated, some of these steps may begin in parallel to development.⁴ Additionally, implementation introduces many challenges that cannot be addressed during development.

The necessary steps of the implementation phase include:

1. Technical Integration: Implementing AI models requires connecting them to disparate HIT systems, including databases, web services, and EMR interfaces. This technical work often involves complex interoperability issues and necessitates a deep understanding of AI models and HIT infrastructure.
2. Prospective Validation: The model needs to be validated in real-world settings after technical integration. Prospective validation ensures the model performs well with real patient data and under actual clinical conditions. Before full-scale deployment, it is essential to confirm model utility and reliability.
3. Workflow Integration: It is crucial to ensure that AI tools are seamlessly integrated into clinical workflows. This involves tailoring the AI model outputs to be easily interpretable and actionable by healthcare professionals. The goal is to support clinical decision-making without adding unnecessary complexity or cognitive load.
4. Monitoring: Once deployed, AI systems require continuous monitoring to ensure they perform as expected. Over time, changes in patient populations, medical practices, and healthcare systems can lead to performance degradation. Continuous monitoring helps identify when a model’s predictions may no longer be reliable, prompting the need for an update or recalibration.
5. Updating: The dynamic nature of healthcare necessitates periodic updates to AI models. Developers must engage with end-users to understand their needs and challenges, ensuring that the AI tools enhance rather than disrupt clinical practice. This ongoing maintenance is vital for keeping AI tools accurate, relevant, and effective in delivering high-quality patient care.

In addition to these steps, successful implementation requires special attention to human factors and systems design. AI models are not used in a vacuum; they must fit into healthcare providers’ existing workflows. Developers must engage with end-users to understand their needs and challenges, ensuring that the AI tools enhance rather than disrupt clinical practice.

Despite their promise, successfully developing, implementing, and periodically updating AI models for healthcare is a challenging engineering task. It requires a collaborative approach, with active involvement from technical experts and healthcare professionals. Development teams can ensure that AI models enhance clinical care and patient outcomes by addressing the technical, human, and workflow considerations.

Where to go from here?

Congratulations! You’ve just completed the 10 super easy steps of developing and implementing AI in healthcare! Well, they may not be super easy, but they are essential and fun (at least, I think they are fun).

This post is the first in a series that catalogs the elements of the AI lifecycle and the relevant infrastructure necessary to support it. Keep an eye out for upcoming posts that cover development and implementation individually in greater detail. We will also discuss the HIT infrastructure required to support the creation and deployment of healthcare AI tools. By the end of this series, you should have a comprehensive understanding of the lifecycle of AI in medicine and the infrastructure and best practices needed to make these systems work effectively.

Some of this content was adapted from the introductory chapter of my doctoral thesis, Machine Learning for Healthcare: Model Development and Implementation in Longitudinal Settings. Hopefully, this series will pique your curiosity and equip you with the knowledge to guide the use of AI in healthcare.

Cheers,
Erkin
Go ÖN Home

1. I adapted this definition from Wikipedia. ↩

2. Another definition adapted from Wikipedia. ↩

3. There’s some disagreement regarding where the border between internal and external validation lies. The details don’t matter too much right now; suffice it to say it’s often contingent on where the data came from: here or there. ↩

4. This has been the traditional healthcare AI/ML development and implementation model. There is some evidence that this approach might need to be revised shortly. ↩