Medical Call Center Nurse Inter-Rater Agreement in Determining “Who should the Telephone Triage Nurse Speak to Next”. Call Prioritization Index (Version 2)
Authors:
David A. Thompson 1
Laurie Whittemore 2
1 Northwestern University, Northwestern University Feinberg School of Medicine
2 Intellicare (SironaHealth), South Portland, Maine
Background: In many larger medical call centers, a non-clinical service representative answers all inbound calls, records patient demographics, and documents the reason for call using the caller’s own words. The call is entered into a call queue and then may be re-ordered by a supervising nurse based upon the call center’s prioritization policy. The purpose of this study is to examine medical call center nurses’ inter-rater agreement in prioritizing telephone triage calls using a revised call prioritization index (CPI v2).
Methods: A random sample of 200 calls was selected from all inbound telephone triage calls (N = 5,520) for a month period. Four nurses were asked to rank calls using a CPI of 1 (Life-Threatening; e.g., choking, cardiac arrest, coma) to 5 (Health Information Request, No Symptoms) to each of the 200 calls using the provided information for each call of age, gender, and reason for call (chief complaint). The nurses were instructed that the CPI answers the question “who should the telephone triage nurse speak to next?”; the nurses were given the CPI as a table containing the ranking (1 to 5), description, and examples.
Results: Statistical analysis included computation of overall inter-rater reliability for the four nurses using a Fleiss’ Kappa statistic. The resulting Kappa value for four nurses was 0.534 (“moderate agreement”). The computations of the Kappa statistic will change as the number of raters change; for two nurses (e.g., nurse 1, nurse 3) the Kappa statistics was 0.689 (“substantial agreement”). Spearman’s rank order correlations were performed to determine if nurses were uniform in their prioritization directionally, even if their rankings were not exactly the same. All bivariate correlations between nurses (paired comparisons of two nurses) were significant (p < 0.01) and showed strong, direct associations (correlation coefficient = 0.679 – 0.774).
Conclusion: Using a call prioritization index (CPI), four nurses achieved moderate agreement ranking 200 telephone triage calls when presented with only the patient age, gender, and a brief reason for call. Individually each nurse prioritizes calls in a manner that is directionally uniform with the other nurses. Encounters in which the triager recommended a lower level of care have the potential for improving patient safety by prioritizing sicker patients and reducing cost by load balancing triager work and optimizing use of limited healthcare resources.
Citation: Thompson DA, Whittemore L. Medical Call Center Nurse Inter-Rater Agreement in Determining “Who should the Telephone Triage Nurse Speak to Next”. Call Prioritization Index (Version 2). Unpublished data and abstract. 2009.
Related References
Brown K, Tabone G, Thompson DA. The impact of telephone triage on healthcare costs – an analysis of caller intent and outcomes. 2018. Abstract and Chart.
Thompson DA, Bolster A, Setnik G. Physician Inter-Rater Agreement in Determining “Who should the Telephone Triage Nurse Speak to Next”. Call Prioritization Index (Version 1). Unpublished data and abstract. 2008.
Thompson DA, Bolster A. Medical Call Center Nurse Inter-Rater Agreement in Determining “Who should the Telephone Triage Nurse Speak to Next”. Call Prioritization Index (Version 1). Unpublished data and abstract. 2008.
Wheeler SQ, Greenberg ME, Mahlmeister L, Wolfe N. Safety of clinical and non-clinical decision makers in telephone triage: a narrative review. J Telemed Telecare. 2015 Sep;21(6):305-22. PubMed Abstract.
Related Documents