Cross-validation strategies under data dependency: An example with anemia prediction in sheep using ocular conjunctiva images.

Abstract

Computer vision techniques have been increasingly used in livestock farming for tasks such as animal identification and health monitoring, but their predictive performance depends on appropriate validation. Validation is critical not only for new animals but also under different management conditions to ensure generalizable predictions across farms. Thus, we evaluated three cross-validation (CV) strategies, random splitting (Rsplit), splitting based on animals (Asplit), and splitting based on farms (Fsplit), to assess deep learning performance in identifying anemic sheep from ocular conjunctiva images. The dataset comprised 1176 images from 186 animals across three farms, captured with a smartphone. Images were segmented using a U-Net algorithm and cropped to retain only the ocular conjunctiva region, which were then input to a VGG19 model. Anemia status was defined using packed cell volume (PCV), with a threshold of 27% for a binary outcome. Prediction metrics included accuracy, precision, recall, F1 score, and specificity. Most metrics, except recall, were higher for Rsplit validation. Accuracy under Asplit was 0.54 ± 0.09, a 24% reduction compared to Rsplit, and F1 score decreased. Under Fsplit, accuracy and F1 score dropped by 34% and 15%, respectively, confirming that predictive ability decreases when models are evaluated under new farm conditions. These results demonstrate that CV strategy strongly impacts model performance: random splits may overestimate accuracy due to repeated measures, while animal- and farm-level splits provide more conservative and realistic estimates. Aligning CV strategy with the dataset structure and deployment scenario is essential for developing robust, generalizable models for real-world veterinary applications.