AI Driven Classification of Immunization-related Clinical Data

The prototype explored two distinct but synergistic approaches to CDSi code classification:

1. LLM-Based Classification

• This approach starts with a patient’s electronic health record in text format.
• Extracts only current conditions (those without an end date) from the record.
• Retrieves a list of CDSi codes and their descriptions stored in Amazon Simple Storage Service (S3).
• Sends both the conditions and the codes to a large language model (LLM) accessed through Amazon Bedrock, with a custom prompt instructing the model to match conditions only when there is clear alignment, avoiding assumptions or inferences.

The model responds with:

• A list of matched CDSi codes
• Corresponding observation titles
• References linking each match to the source condition in the patient record

2. SNOMED-to-CDSi Mapping

Recognizing that many EHRs include SNOMED clinical terms, a second approach uses SNOMED codes as a bridge to CDSi classification:

• Direct Matching: SNOMED codes are extracted from structured sections of the patient’s CDA-formatted health document, including conditions and surgeries. These codes are then matched to CDSi codes using a lookup table stored in Amazon DynamoDB.
• AI-Based Extraction: For conditions buried in unstructured text, AWS Comprehend Medical extracts SNOMED codes, which are then queried against the same DynamoDB mapping table to return applicable CDSi codes.

These two methods give health agencies flexibility depending on the structure and quality of incoming patient data—enabling a robust CDSi classification pipeline across diverse EHR formats.

Experiments & Abandoned Approaches

The team also explored several innovative but ultimately unused paths, which are documented to avoid future repetition and to share learnings with the broader community:

Medical BERT Similarity Analysis

The team tested several pretrained medical BERT models by comparing semantic similarity between patient condition terms and CDSi observations across relation types like synonymy, causality, and comorbidity. Surprisingly, unrelated or even opposite terms often scored higher than clinically relevant ones, revealing a disconnect between linguistic similarity and actual clinical relationships. As a result, this approach  was deemed unreliable for downstream decision-making.

Context Expansion via Hyde AI Mechanism

Another early experiment used a Hyde AI-inspired approach to enrich LLM prompts with additional concepts—like likely medications or symptoms—based on each CDSi observation title. While this enriched context showed some promise, the model frequently made incorrect assumptions about clinical relationships, leading to misclassifications. Due to its unpredictability and lack of clinical reasoning, this method was ultimately set aside.