Abstract The evolution of agriculture towards intensive farming leads to an increasing demand for animal identification associated with high traceability, driven by the need for quality control and welfare management in agricultural animals. Automatic identification of individual animals is an important step to achieve individualised care in terms of disease detection and control, and improvement of the food quality. For example, as feeding patterns can differ amongst pigs in the same pen, even in homogenous groups, automatic registration shows the most potential when applied to an individual pig. In the EU for instance, this capability is required for certification purposes. Although the RFID technology has been gradually developed and widely applied for this task, chip implanting might still be time-consuming and costly for current practical applications. In this paper, a novel framework composed of computer vision algorithms, machine learning and deep learning techniques is proposed to offer a relatively low-cost and scalable solution of pig recognition. Firstly, pig faces and eyes are detected automatically by two Haar feature-based cascade classifiers and one shallow convolutional neural network to extra high-quality images. Secondly, face recognition is performed by employing a deep convolutional neural network. Additionally, class activation maps generated by grad-CAM and saliency maps are utilised to visually understand how the discriminating parameters have been learned by the neural network. By applying the proposed approach on 10 randomly selected pigs filmed in farm condition, the proposed method demonstrates the superior performance against the state-of-art method with an accuracy of 83% over 320 testing images. The outcome of this study will facilitate the real-application of AI-based animal identification in swine production.