Objectives The purpose of this study was to assess the inter-rater reliability and intra-rater reliability of medical record review for the detection of hospital adverse events.
Methods We conducted two stages retrospective medical records review of a random sample of 96 patients from one acute-care general hospital. The first stage was an explicit patient record review by two nurses to detect the presence of 41 screening criteria (SC). The second stage was an implicit structured review by two physicians to identify the occurrence of adverse events from the positive cases on the SC. The inter-rater reliability of two nurses and that of two physicians were assessed. The intra-rater reliability was also evaluated by using test-retest method at approximately two weeks later.
Results In 84.2% of the patient medical records, the nurses agreed as to the necessity for the second stage review (kappa, 0.68; 95% confidence interval [CI], 0.54 to 0.83). In 93.0% of the patient medical records screened by nurses, the physicians agreed about the absence or presence of adverse events (kappa, 0.71; 95% CI, 0.44 to 0.97). When assessing intra-rater reliability, the kappa indices of two nurses were 0.54 (95% CI, 0.31 to 0.77) and 0.67 (95% CI, 0.47 to 0.87), whereas those of two physicians were 0.87 (95% CI, 0.62 to 1.00) and 0.37 (95% CI, -0.16 to 0.89).
Conclusions In this study, the medical record review for detecting adverse events showed intermediate to good level of inter-rater and intra-rater reliability. Well organized training program for reviewers and clearly defining SC are required to get more reliable results in the hospital adverse event study.
Summary
Citations
Citations to this article as recorded by
The Korea National Patient Safety Incidents Inquiry Survey: Feasibility of Medical Record Review for Detecting Adverse Events in Regional Public Hospitals Min Ji Kim, Hee Jung Seo, Hong Mo Koo, Minsu Ock, Jee-In Hwang, Sang-Il Lee Journal of Patient Safety.2022; 18(5): 389. CrossRef
The Korea National Patient Safety Incidents Inquiry Survey: Characteristics of Adverse Events Identified Through Medical Records Review in Regional Public Hospitals Min Ji Kim, Hee Jung Seo, Hong Mo Koo, Minsu Ock, Jee-In Hwang, Sang-Il Lee Journal of Patient Safety.2022; 18(5): 382. CrossRef
Feasibility of Capturing Adverse Events From Insurance Claims Data Using International Classification of Diseases, Tenth Revision, Codes Coupled to Present on Admission Indicators Juyoung Kim, Eun Young Choi, Won Lee, Hae Mi Oh, Jeehee Pyo, Minsu Ock, So Yoon Kim, Sang-il Lee Journal of Patient Safety.2022; 18(5): 404. CrossRef
Quality of end-of-life communication in 2 high-risk ICU cohorts: a retrospective cohort study Tammy L. Pham, Allan Garland CMAJ Open.2021; 9(2): E570. CrossRef
A Systematic Review of Methods for Medical Record Analysis to Detect Adverse Events in Hospitalized Patients Dorthe O. Klein, Roger J.M.W. Rennenberg, Richard P. Koopmans, Martin H. Prins Journal of Patient Safety.2021; 17(8): e1234. CrossRef
Patient Safety Incidents Reported by the General Public in Korea: A Cross-Sectional Study Minsu Ock, Min-Woo Jo, Eun Young Choi, Sang-Il Lee Journal of Patient Safety.2020; 16(2): e90. CrossRef
Perceptions of Hospital Health Information Managers Regarding Present on Admission Indicators in Korea: A Qualitative Study Jee-Hee Pyo, Eun-Young Choi, Hae-Mi Oh, Won Lee, Ju-Young Kim, Min-Su Ock, So-Yoon Kim, Sang-Il Lee Quality Improvement in Health Care.2020; 26(1): 23. CrossRef
Variation between hospitals and reviewers in detection of adverse events identified through medical record review in Korea Sukyeong Kim, Ho Gyun Shin, A E Jeong Jo, Ari Min, Minsu Ock, Jee-In Hwang, Youngjin Jeong, Moon Sung Park, Jong Bouk Lee, Tae I K Chang, Eunhyang Song, Heungseon Kim, Sang-Il Lee International Journal for Quality in Health Care.2020; 32(8): 495. CrossRef
The Harvard medical practice study trigger system performance in deceased patients Dorthe O. Klein, Roger J. M. W. Rennenberg, Richard P. Koopmans, Martin H. Prins BMC Health Services Research.2019;[Epub] CrossRef
Optimising Hepatitis C care in an urban Aboriginal and Torres Strait Islander primary health care clinic Prabha Lakhan, Deborah Askew, Noel Hayman, Leigh‐anne Pokino, Cheryl Sendall, Paul J. Clark Australian and New Zealand Journal of Public Health.2019; 43(3): 228. CrossRef
Patient and hospital characteristics that influence incidence of adverse events in acute public hospitals in Portugal: a retrospective cohort study Paulo Sousa, António Sousa Uva, Florentino Serranheira, Mafalda Sousa Uva, Carla Nunes International Journal for Quality in Health Care.2018; 30(2): 132. CrossRef
Interpregnancy weight change and adverse pregnancy outcomes: a systematic review and meta-analysis Eugene Oteng-Ntim, Sofia Mononen, Olga Sawicki, Paul T Seed, Debra Bick, Lucilla Poston BMJ Open.2018; 8(6): e018778. CrossRef