To achieve rapid and low-cost risk identification of veterinary drug residues in raw milk, this study proposed a machine learning and federated learning based method for rapidly detecting veterinary drug residues in raw milk, focusing on the routine detection indicators of raw milk and veterinary drug residue data in various regions of China provided by a large dairy enterprise from 2022 to 2024. This model served as an effective supplement to traditional chemical detection methods. To address the data imbalance characteristic where positive samples of veterinary drug residues were far fewer than negative samples, a combined strategy of SMOTE upsampling and random downsampling was adopted to optimize the training set. Through comparative experiments of various machine learning algorithms such as Random Forest, Decision Tree, Bayesian Network, eXtreme Gradient Boosting, and Support Vector Machine, eXtreme Gradient Boosting model had the optimal comprehensive performance in both binary-classification and multi-classification prediction of veterinary drug residues. Feature importance analysis using the eXtreme Gradient Boosting model showed that, “region” feature was the primary factor affecting the status of veterinary drug residues, revealing the overall impact of regional differences on the quality and safety of raw milk. By constructing a client-side learning model based on regional division, this study introduced a federated learning framework in eXtreme Gradient Boosting to achieve dual effects of data privacy protection and prediction accuracy comparable to that of centralized learning, and also improve model stability. This research could provide a fast, low-cost, data-driven risk identification paradigm and warning mechanism for the detection of veterinary drug residues in raw milk, serving as an effective supplement to chemical detection technologies and a new solution for food safety assurance.