Comparison of Diagnostic Recommendations from Individual Physicians versus the Collective Intelligence of Multiple Physicians in Ambulatory Cases Referred for Specialist Consultation

Abstract

Background

Studies report higher diagnostic accuracy using the collective intelligence (CI) of multiple clinicians compared with individual clinicians. However, the diagnostic process is iterative, and unexplored is the value of CI in improving clinical recommendations leading to a final diagnosis.

Methods

To compare the appropriateness of diagnostic recommendations advised by individual physicians versus the CI of physicians, we entered actual consultation requests sent by primary care physicians to specialists onto a web-based CI platform capable of collecting diagnostic recommendations (next steps for care) from multiple physicians. We solicited responses to 35 cases (12 endocrinology, 13 gynecology, 10 neurology) from ≥3 physicians of any specialty through the CI platform, which aggregated responses into a CI output. The primary outcome was the appropriateness of individual physician recommendations versus the CI output recommendations, using recommendations agreed upon by 2 specialists in the same specialty as a gold standard. The secondary outcome was the recommendations’ potential for harm.

Results

A total of 177 physicians responded. Cases had a median of 7 respondents (interquartile range: 5–10). Diagnostic recommendations in the CI output achieved higher levels of appropriateness (69%) than recommendations from individual physicians (45%; χ² = 5.95, P = 0.015). Of the CI recommendations, 54% were potentially harmful, as compared with 41% of individuals’ recommendations (χ² = 2.49, P = 0.11).

Limitations

Cases were from a single institution. CI was solicited using a single algorithm/platform.

Conclusions

When seeking specialist guidance, diagnostic recommendations from the CI of multiple physicians are more appropriate than recommendations from most individual physicians, measured against specialist recommendations. Although CI provides useful recommendations, some have potential for harm. Future research should explore how to use CI to improve diagnosis while limiting harm from inappropriate tests/therapies.