Share on Facebook
Share on Twitter
Share on LinkedIn
Share by Email
Print
Machine-learning algorithms that incorporate administrative data and deep learning may be effective for predicting risk for opioid overdose and for identifying patients at low risk for overdose, according to a study published in JAMA Network Open.
Cancer-free fee-for-service Medicare beneficiaries who filled ≥1 opioid prescription between 2011 and 2015 were included in this prognostic study. The occurrence of fatal or nonfatal opioid overdose, which was defined in 3-month windows following the initial prescription, was identified in each patient.
A total of 268 potential predictors (eg, sociodemographics, health status, patterns of opioid use, and practitioner- and regional-level factors) were evaluated for 3-month periods, starting before opioid prescription. The study’s main outcomes included opioid overdose episodes identified with inpatient and emergency department claims. Risk prediction and risk stratification were the primary and secondary goals, respectively, of the analysis.
In the study, 3 samples were used (n=186,686 for each): the training sample to develop algorithms, the testing sample to refine algorithms, and the validation sample to evaluate the prediction performance of the developed algorithm. Multivariate logistic regression, least absolute shrinkage and selection operator-type regression (LASSO), random forest (RF), gradient boosting machine (GBM), and deep neural network (DNN) were the 5 machine-learning approaches used for developing and testing prediction algorithms for opioid overdose.
For the prediction of opioid overdose in the validation sample, both the DNN (C statistic, 0.91; 95% CI, 0.88-0.93) and GBM (C statistic, 0.90; 95% CI, 0.87-0.94) algorithms were superior to the LASSO (C statistic, 0.84; 95% CI, 0.80-0.89), RF (C statistic, 0.80; 95% CI, 0.75-0.84), and MLR (C statistic, 0.75; 95% CI, 0.69-0.80) algorithms. The DNN had a sensitivity and specificity of 92.3% and 75.7%, respectively, a positive predictive value of 0.18%, and a negative predictive value of 99.9%.
The deep learning method was effective for stratifying patients into low- (76.2%), medium- (18.6%), and high-risk (5.2%) groups. Only 1 patient of 10,000 in the low-risk subgroup experienced an overdose episode. The majority of overdoses (>90%) occurred in patients classified as high- and medium-risk.
Study limitations included the lack of data on access to opioids and overdoses outside the medical setting, as well as the sole inclusion of Medicare beneficiaries.
“This study demonstrates the feasibility and potential of machine-learning prediction models with routine administrative claims data available to payers. These models have high C statistics and good prediction performance and appear to be valuable tools for more accurately and efficiently identifying individuals at high risk of opioid overdose,” concluded the study authors.
Reference
Lo-Ciganic WH, Huang JL, Zhang HH, et al. Evaluation of machine-learning algorithms for predicting opioid overdose risk among Medicare beneficiaries with opioid prescriptions. JAMA Netw Open. 2019;2(3):e190968.
