Risk stratification of potential drug interactions involving common over-the-counter medications and herbal supplements by a large language model.

Abstract

<h4>Background</h4>As polypharmacy, the use of over-the-counter (OTC) drugs, and herbal supplements becomes increasingly prevalent, the potential for adverse drug-drug interactions (DDIs) poses significant challenges to patient safety and health care outcomes.<h4>Objective</h4>This study evaluates the capacity of Generative Pre-trained Transformer (GPT) models to accurately assess DDIs involving prescription drugs (Rx) with OTC medications and herbal supplements.<h4>Methods</h4>Leveraging a popular subscription-based tool (Lexicomp), we compared the risk ratings assigned by these models to 43 Rx-OTC and 30 Rx-herbal supplement pairs.<h4>Results</h4>Our findings reveal that all models generally underperform, with accuracies below 50% and poor agreement with Lexicomp standards as measured by Cohen's kappa. Notably, GPT-4 and GPT-4o demonstrated a modest improvement in identifying higher-risk interactions compared to GPT-3.5.<h4>Conclusion</h4>These results highlight the challenges and limitations of using off-the-shelf large language models for guidance in DDI assessment.