The application of machine learning (ML) technologies in health care is expected to improve care delivery and patient outcomes. However, there are no best practices for designing these technologies for use in clinical settings. To explore user needs and design requirements for a user interface of a ML risk prediction tool in development, we consulted with subject matter experts and physicians. We explored physician expectations of using a ML tool in clinical practice and their preferences on designs. Our process revealed physician perspectives on trusting a ML tool and opportunities to design for these considerations, while navigating ambiguity in the tool’s outputs.
AtanasovP.GauthierA.LopesR. (2018). PMU5: Applications of artificial intelligence technologies in healthcare: A systematic literature review. Value in Health, 21(2), S84. doi:10.1016/j.jval.2018.07.629
2.
BaerlocherM. O. (2007). Canada’s slow adoption of new technologies adds burden to health care system. Canadian Medical Association Journal, 176, 616. Retrieved from http://www.cmaj.ca/content/176/5/616
3.
BauerM. S.DamschroderL.HagedornH.SmithJ.KilbourneA. M. (2015). An introduction to implementation science for the non-specialist. BMC Psychology, 3(1), Article number 32. doi:10.1186/s40359-015-0089-9
4.
BeamA. L.KohaneI. S. (2016). Translating artificial intelligence into clinical care. JAMA Journal of American Medical Association, 316, 2368–2369. doi:10.1001/jama.2016.17217
5.
Couzin-FrankelJ. (2019). Medicine contends with how to use artificial intelligence, barriers include lack of reproducibility across hospitals. Science, 363, 1119–1120.
6.
DzindoletM. T.PetersonS. A.PomrankyR. A.PierceL. G.BeckH. P. (2003). The role of trust in automation reliance. International Journal of Human–Computer Studies, 58, 697–718.
7.
GuidottiR.MonrealeA.RuggieriS.TuriniF.GiannottiF.PedreschiD. (2018). A survey of methods for explaining black box models. ACM Computing Surveys (CSUR), 51(5), 93.
HintonG. (2018). Deep learning: A technology with the potential to transform health care. JAMA Journal of American Medical Association, 320, 1101–1102. doi:10.1001/jama.2018.11100
11.
JiangF.JiangY.ZhiH.DongY.LiH.MaS.. . . WangY. (2017). Artificial intelligence in healthcare: Past, present and future. Stroke and Vascular Neurology, 2, 230–243.
12.
JieM. A.CollinsG. S.SteyerbergE. W.VebakelJ. Y.van CalsterB. (2019). A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. Journal of Clinical Epidemiology, 110, 12–22.
MadhavanP.WiegmannD. A. (2007). Similarities and differences between human-human and human-automation trust: An integrative review. Theoretical Issues in Ergonomics Science, 8, 277–301.
18.
MillerD. D.BrownE. W. (2018). Artificial intelligence in medical practice: The question to the answer?American Journal of Medicine, 131, 129–133. doi:10.1016/j.amjmed.2017.10.035
NaylorC. D. (2018). On the prospects for a (deep) learning health care system. JAMA Journal of American Medical Association, 320, 1099–1100. doi:10.1001/jama.2018.11103
21.
NorrieJ. (2018). The challenge of implementing AI models in the ICU. Lancet, 6, 886–888.
22.
OberijeC.NalbantovG.DekkerA.BoersmaL.BorgerJ.ReymenB.. . . LambinP. (2014). A prospective study comparing the predictions of doctors versus models for treatment outcome of lung cancer patients: A step toward individualized care and shared decision making. Radiotherapy and Oncology, 112, 37–43. doi:10.1016/j.radonc.2014.04.012
ShahidN.RapponT.BertaW. (2019). Applications of artificial neural networks in health care organizational decision-making: A scoping review. PLoS One, 14, e0212356.
VergheseA.ShahN. H.HarringtonR. A. (2018). What this computer needs is a physician: Humanism and artificial intelligence. JAMA Journal of American Medical Association, 319, 19–20. doi:10.1001/jama.2017.19198
