Evaluation of videoendoscopic examinations of arytenoid function in the 2-year-old Thoroughbred: Can we all agree?

Abstract

BACKGROUND: Upper respiratory tract (URT) endoscopic examination is a routine part of prepurchase examinations. Discrepancies have been documented in the assessment of arytenoid function grades (AFG) between veterinarians. OBJECTIVES: To document intra- and interobserver agreement for a population of multi-experience level veterinarians for assessment of AFG of 2-year-old Thoroughbreds. STUDY DESIGN: Observational cohort study. METHODS: One-hundred and fourteen URT videoendoscopic examinations were evaluated by 10 veterinarians. Veterinarians were categorised based on experience, into five groups, each group with two veterinarians. Arytenoid function was graded using the Havemeyer ordinal scale and then reclassified by the authors dichotomously into 'meets conditions of sale' (MCS) or 'does not meet conditions of sale'(DNMCS). Interobserver agreement of arytenoid function was assessed across all 10 veterinarians using Fleiss' kappa and between veterinarians of similar experience levels in the five subgroups using Cohen's unweighted (k) and Cohen's linear weighted kappa (Ck). Intraobserver agreement was similarly calculated for each reviewer using 22 repeated video clips. RESULTS: Overall interobserver agreement using ordinal scales was fair (k = 0.27, 95% CI 0.22-0.31) to moderate (mean weighted Ck = 0.57, 95% CI 0.46-0.69) depending on statistical methodology used. Using the dichotomous classification, interobserver agreement was good (k = 0.7, 95% CI 0.63-0.77). Overall intraobserver agreement using ordinal scales was fair (mean k = 0.26, 95% CI 0.14-0.38) to good (mean Ck = 0.61, 95% CI 0.50-0.71); and for the dichotomous reclassification it was good (k = 0.73, 95% CI 0.59-0.87). Experience level differences were identified. MAIN LIMITATIONS: The low number of veterinarians in each experience subgroup. CONCLUSIONS: Subjectivity exists in arytenoid function grading, despite the existence of a well-defined scale. Agreement variation exists depending on the grading scales and statistical methods used for analysis. Future studies pre- and post veterinarian training are indicated to determine if agreement can be improved.