Academic Journal of Science and Technology 
ISSN: 2771-3032 | Vol. 9, No. 1, 2024 

 

226 

Image	Recognition	for	Fruit‐Picking	Robots	
Jianxin Li1, a, Yibing Zhou1, b, Zimo Gao1, c 

1School of Electronic Information Engineering, Liaoning Technical University, China 
a2767101156@qq.com, b2219078335@qq.com, cGZM318426@163.com 

 

Abstract: This summary presents the solutions to the five questions posed in the competition regarding apple image 
recognition. The objective is to establish a robust apple image recognition model that accurately counts the number of apples, 
estimates their positions, determines their maturity state, calculates their masses, and recognizes them based on provided image 
datasets. The specific answers to each question are as follows: Using the image dataset of harvest-ready apples provided in 
Attachment 1, an image feature extraction method is employed to establish a mathematical model for counting the number of 
apples in each image. The distribution of all apples in Attachment 1 is then depicted in a histogram. The precise number of apples 
in each image is determined, enabling a comprehensive understanding of the dataset. Utilizing the image dataset of harvest-ready 
apples provided in Attachment 1, the positions of the apples in each image are identified. The left bottom corner of the image 
serves as the coordinate origin, and a two-dimensional scatter diagram is created, illustrating the geometric coordinates of all 
apples in Attachment 1. This analysis facilitates the visualization of the spatial distribution of apples within the dataset. Based 
on the image dataset of harvest-ready apples provided in Attachment 1, a mathematical model is developed to assess the maturity 
state of apples in each image. By applying this model, a histogram is generated, illustrating the distribution of maturity levels 
across all apples in Attachment 1. This analysis provides insights into the overall maturity status of the dataset. 

Keywords: Asia and Pacific Mathematical Contest in Modeling, fruit-picking robots, apple recognition, image recognition, 
labor shortage, obstacle identification, maturity estimation, mass estimation, dataset. 

 

1. Introduction 
Apple harvesting faces significant challenges due to the 

labor-intensive nature of the process and a shortage of skilled 
workers during peak seasons. To address this, China has 
invested in the development of apple-picking robots since 
2011. However, the widespread adoption of these robots faces 
hurdles, particularly in complex orchard environments where 
precise identification of obstacles and fruits is crucial for 
efficient and damage-free harvesting. Recognizing the 
limitations, this competition focuses on establishing an 
advanced apple image recognition model. The goal is to 
achieve high recognition rates, fast processing speeds, and 
accuracy in tasks such as counting apples, estimating 
positions, assessing maturity levels, and estimating masses. 
The provided image datasets (Attachment 1 and Attachment 
2) serve as the foundation for developing and validating these 
models. 

2. Problem Analysis 
Question 1: Counting Apples: 
Problem Analysis: 
The task involves extracting image features from the 

provided dataset of harvest-ready apples, establishing a 
mathematical model, and counting the number of apples in 
each image. The primary challenge lies in developing an 
algorithm that accurately identifies individual apples amid 
variations in shape, color, and occlusions. 

Model Hypotheses: 
Each apple in the image can be represented as a distinct 

object based on color, shape, and size features. 
Mathematical model: Utilize image processing techniques, 

possibly employing contour detection and object 
segmentation, to identify and count individual apples 
accurately. 

Histogram Distribution: Visualizing the distribution of 
counted apples across all images can provide insights into the 
overall dataset. 

Question 2: Estimating the Positions of Apples: 
Problem Analysis: 
Identifying the spatial positions of apples within each 

image is essential for robot guidance. The challenge lies in 
developing a model that accurately captures the geometric 
coordinates of apples, considering variations in orientation 
and location within the image. 

Model Hypotheses: 
Geometric Coordinates: The position of each apple can be 

determined with respect to the left bottom corner of the image. 
Two-Dimensional Scatter Diagram: Visualizing the 

distribution of apple positions can reveal patterns and aid in 
assessing the model's accuracy. 

Question 3: Estimating the Maturity State of Apples: 
Problem Analysis: 
Determining the maturity state of apples is crucial for 

optimizing harvesting times. The challenge involves 
developing a model that accurately assesses the maturity level 
based on visual cues in the images. 

Model Hypotheses: 
Color and Texture Analysis: Maturity can be inferred from 

color changes and surface textures in the images. 
Mathematical Model: Develop an algorithm that quantifies 

these visual cues to estimate the maturity state. 
Histogram of Maturity Distribution: Visualizing the 

distribution of maturity levels provides insights into the 
overall dataset. 

Question 4: Estimating the Masses of Apples: 
Problem Analysis: 
Estimating the masses of apples is vital for logistical 

planning and quality control. The challenge involves 
developing a model that accurately correlates two-
dimensional areas of apples with their masses. 



 

227 

Model Hypotheses: 
Two-Dimensional Area Calculation: Use image processing 

techniques to calculate the area of each apple. 
Mass Estimation Model: Develop a correlation between the 

calculated areas and actual masses. 
Histogram of Mass Distribution: Visualizing the 

distribution of estimated masses aids in quality assessment. 
Question 5: The Recognition of Apples: 
Problem Analysis: 
Recognizing apples in a broader dataset involves training 

an apple recognition model using a separate dataset. 
Challenges include distinguishing apples from other fruits 
with similar visual features. 

3. Problem Assumption 
Feature Extraction: Extract unique features from apple 

images to train the recognition model. 
Machine Learning: Utilize machine learning algorithms, 

possibly convolutional neural networks (CNNs), to train the 
model. 

Distribution Histogram: Visualize the distribution of 
identified apples to assess the model's performance. 

This comprehensive approach aims to tackle the 
complexities of apple image recognition, providing valuable 
insights for the development of efficient and accurate robotic 
harvesting systems. 

4. Symbol Description 

Symbol Explain 
 X data matrix 
A matrix 
B load matrix 

π                         amount 
𝑈                        binding 

5. Establishment and Solution of Model 

5.1. Question one 
Image Preprocessing: Preprocess the images to enhance 

their quality and remove any noise that might interfere with 
the apple detection. Common preprocessing techniques 
include resizing, denoising, and contrast adjustment. 

Apple Detection: Apply an object detection algorithm or 
technique to locate and identify apples in the images. This can 
be achieved using techniques like template matching, edge 
detection, or machine learning-based object detection 
algorithms such as Faster R-CNN or YOLO. 

Apple Counting: Once the apples are detected, count the 
number of individual apple instances in each image. This can 
be done by applying a counting algorithm, such as connected 
component analysis or contour detection, to identify and 
count distinct objects representing apples. 

Histogram Generation: After counting the apples in each 
image, create a histogram to visualize the distribution of apple 
counts across all the images in Attachment 1. The x-axis of 
the histogram represents the number of apples, while the y-
axis represents the frequency or occurrence of each count. 

5.2. Question two 
To estimate the positions of apples in each image from the 

provided image dataset, we can use computer vision 
techniques. Specifically, we can use object detection 

algorithms to identify the apples and determine their 
coordinates in the image. 

One popular object detection algorithm is the YOLO (You 
Only Look Once) algorithm, which can detect objects in an 
image and provide their bounding boxes, including the 
coordinates of the top-left and bottom-right corners of each 
bounding box. 

 

 
 
To proceed with this task, we can follow these steps: 
Preprocess the images: Before applying any object 

detection algorithm, we may need to preprocess the images to 
enhance their quality and make them more suitable for 
analysis. This can include steps such as resizing, 
normalization, and filtering to remove noise. 

Train or use a pre-trained object detection model: We can 
either train an object detection model on the provided image 
dataset specifically to detect apples or use a pre-trained model 
that has been trained on a large dataset of images and can 
detect various objects, including apples. 

Apply the object detection model: Once we have a trained 
or pre-trained model, we can apply it to the images in the 
dataset to detect apples and obtain their bounding boxes. 

Extract the coordinates of apples: From the bounding boxes 
obtained in the previous step, we can extract the coordinates 
of the top-left and bottom-right corners of each apple. 

Convert the coordinates to a common scale: Since the 
images may have different sizes and aspect ratios, we need to 
convert the coordinates of the apples to a common scale, such 
as normalizing them to a range of [0, 1] based on the image 
dimensions. 

Create a scatter diagram: Finally, we can use a plotting 
library to create a two-dimensional scatter diagram of the 
geometric coordinates of all apples in Attachment 1, with the 
left bottom corner of each image as the coordinate origin. 

5.3. Question three 
Background: 
Estimating the maturity state of apples is a critical aspect 

of optimizing harvest times and ensuring the quality of the 
produce. Visual cues such as color changes and surface 
textures are indicative of maturity levels. In this question, the 
objective is to establish a mathematical model to quantify 
these visual cues, calculate the maturity of apples in each 
image, and visualize the overall maturity distribution through 
a histogram. 

Problem Analysis: 
The challenge involves translating visual information, such 

as color variations and texture changes, into a quantifiable 
metric representing the maturity of apples. Developing an 
accurate model requires a robust understanding of how these 
visual cues correlate with the actual maturity state. 

Model Hypotheses: 



 

228 

1. Color Analysis: 
Different stages of apple maturity exhibit characteristic 

color changes. 
Color spaces such as RGB or HSV can be leveraged to 

quantify color variations. 
The model should assign higher values to colors indicative 

of ripe apples. 
2. Texture Analysis: 
Maturity influences surface textures, such as smoothness or 

roughness. 
Texture analysis techniques, like Gabor filters or local 

binary patterns, can be employed. 
The model should capture texture variations associated 

with different maturity levels. 
3. Combined Metric: 
Combining color and texture metrics to create a 

comprehensive maturity index. 
Weighting factors may be applied based on the significance 

of color and texture in maturity estimation. 
Mathematical Model: 
Adjusting the weighting factors allows fine-tuning the 

model based on the importance of color and texture in the 
specific dataset. 

Calculation Procedure: 
1. Color Calculation: 
Convert image to a suitable color space (e.g., RGB to HSV). 
Quantify color variations using chosen metrics. 
2. Texture Calculation: 
Apply texture analysis techniques to extract relevant 

features. 
Quantify texture variations using chosen metrics. 
3. Maturity Calculation: 
Combine color and texture metrics using the mathematical 

model. 
Assign each apple a maturity value. 
4. Histogram Visualization: 
Create a histogram representing the distribution of maturity 

levels. 
Bins in the histogram represent ranges of maturity values. 
The histogram provides insights into the overall maturity 

distribution. 
Conclusion: 
By developing and implementing this mathematical model, 

we aim to provide a quantitative measure of apple maturity, 
enabling efficient and accurate assessment across the entire 
dataset. The histogram visualization will offer a 
comprehensive view of maturity distribution, aiding in further 
analysis and decision-making in apple harvesting processes. 

 

 

5.4. Question four 
Background: 
Estimating the masses of apples is a crucial aspect of 

quality control and logistical planning in the apple harvesting 
process. In this question, the objective is to calculate the two-
dimensional area of apples in each image, use this information 
to estimate the masses of the apples, and finally, visualize the 
distribution of mass through a histogram. 

Problem Analysis: 
The challenge involves correlating the two-dimensional 

area of apples in images with their actual masses. The 
assumption is that the area provides a representative measure 
of the size of each apple, and this information can be used to 
estimate the masses. 

Model Hypotheses: 
1. Two-Dimensional Area Calculation: 
Use image processing techniques to calculate the area of 

each apple. 
The area can be determined using pixel counts or geometric 

methods. 
The bottom left corner of the image serves as the coordinate 

origin. 
2. Mass Estimation Model: 
Establish a correlation between the calculated two-

dimensional area and actual masses. 
The relationship may be nonlinear, requiring curve fitting 

or machine learning regression. 
Mathematical Model: 
Calculation Procedure: 
1. Two-Dimensional Area Calculation: 
Employ image processing techniques to calculate the area 

of each apple. 
Pixel counts or geometric methods can be used depending 

on the specific requirements. 
2. Mass Estimation: 
Use the established mathematical model to estimate the 

mass of each apple. 
3. Histogram Visualization: 
Create a histogram representing the distribution of 

estimated masses. 
Bins in the histogram represent ranges of mass values. 
The histogram provides insights into the overall mass 

distribution. 
Conclusion: 
By implementing this mathematical model, we aim to 

estimate the masses of apples based on their two-dimensional 
areas, providing a quantitative measure that can be useful for 
quality assessment and logistical planning. The resulting 



 

229 

histogram visualization offers a comprehensive view of the 
distribution of estimated masses across the dataset. 

 

 

 

5.5. Question five 
Image Preprocessing: Preprocess the images in 

Attachments 2 and 3 to enhance their quality, normalize the 
colors, and resize them to a consistent size. This step ensures 
that the images are in a suitable format for feature extraction 
and model training. 

Feature Extraction: Extract relevant features from the 
preprocessed images. Common approaches include using 
deep learning-based methods like convolutional neural 
networks (CNNs) to automatically learn discriminative 
features from the images. This step aims to capture the 
distinctive characteristics of the apple images. 

Dataset Split: Split the dataset in Attachment 2 into training 
and validation sets. The training set will be used to train the 
apple recognition model, while the validation set will be used 
to evaluate its performance. 

Model Training: Train an apple recognition model using 
the extracted image features and the labeled apple images in 
the training set. There are various architectures and 
frameworks available for training such models, including 
CNNs such as VGG, ResNet, or custom-designed models. 
The model should be trained to classify and assign ID 
numbers to the apple images. 

Model Evaluation: Evaluate the trained model using the 
validation set to assess its performance and fine-tune 
hyperparameters if necessary. Common evaluation metrics 
include accuracy, precision, recall, and F1 score. 

Apple Identification: Apply the trained model to the apple 
images in Attachment 3 to identify and assign ID numbers to 
each apple image. This can be done by feeding the 
preprocessed images into the trained model and obtaining the 

predicted labels or ID numbers. 
Distribution Histogram: Create a histogram of the ID 

numbers of all the apple images in Attachment 3 to visualize 
their distribution. The x-axis of the histogram represents the 
ID numbers, while the y-axis represents the frequency or 
occurrence of each ID number. 

% Load and preprocess the training images 

trainingImageFiles = dir('path_to_training_images/*.jpg'); % 

Replace 'path_to_training_images' with the actual folder path 

numTrainingImages = length(trainingImageFiles); 

for i = 1:numTrainingImages 

    trainingImage = 

imread(fullfile(trainingImageFiles(i).folder, 

trainingImageFiles(i).name)); 

    % Apply preprocessing steps (e.g., resizing, color 

normalization) 

    % ... 

     

    % Extract image features 

    trainingFeatures(i, :) = 

extractFeatures(trainingImage); % Replace 'extractFeatures' 

with the appropriate feature extraction method 

end 

% Create labels for the training images 

trainingLabels = [1 2 1 3 2]; % Replace with the actual 

labels for the training images 

% Train the apple recognition model 

model = trainModel(trainingFeatures, trainingLabels); % 

Replace 'trainModel' with the appropriate model training 

function 

% Load and preprocess the apple images for identification 

testImageFiles = dir('path_to_test_images/*.jpg'); % 

Replace 'path_to_test_images' with the actual folder path 

numTestImages = length(testImageFiles); 

for i = 1:numTestImages 

    testImage = imread(fullfile(testImageFiles(i).folder, 

testImageFiles(i).name)); 

    % Apply preprocessing steps (e.g., resizing, color 

normalization) 

    % ... 

     

    % Extract image features 

    testFeatures(i, :) = extractFeatures(testImage); % 

Replace 'extractFeatures' with the appropriate feature 

extraction method 

     

    % Apply the trained model for apple identification 

    predictedLabel = identifyApple(model, 

testFeatures(i, :)); % Replace 'identifyApple' with the 

appropriate prediction function 

     

    % Store the predicted label or ID number 

    predictedLabels(i) = predictedLabel; 



 

230 

end 

% Generate Distribution Histogram 

histogram(predictedLabels); 

xlabel('ID Number'); 

ylabel('Frequency'); 

title('Distribution of Apple ID Numbers'); 

% Function to train the apple recognition model (example 

using a simple SVM classifier) 

function model = trainModel(features, labels) 

    % Train the model using the features and labels 

    % ... 

    model = fitcecoc(features, labels); % Replace with the 

appropriate model training function 

end 

 

% Function to identify apples using the trained model 

(example using a simple SVM classifier) 

function predictedLabel = identifyApple(model, 

testFeatures) 

    % Apply the trained model to predict the label or ID 

number 

    % ... 

    predictedLabel = predict(model, testFeatures); % 

Replace with the appropriate prediction function 

end 

6. Evaluation and Extension of The 
Model 

Model Overview: 
The proposed apple image recognition model addresses the 

challenges in automating apple picking by analyzing and 
extracting features from labeled fruit images. It aims to 
achieve a high recognition rate, fast speed, and accuracy in 
counting apples, estimating their positions, determining 
maturity levels, estimating masses, and recognizing different 
types of apples. The model consists of several key 
components: 

Image Feature Extraction: The model employs advanced 
image processing techniques to extract relevant features from 
the provided image dataset. This step helps in capturing 
important characteristics of the apples, such as color, shape, 
texture, and size. 

Mathematical Modeling: Mathematical models are 
established to perform specific tasks such as counting apples, 
estimating positions, assessing maturity levels, and 
estimating masses. These models utilize the extracted image 
features and relevant parameters to make accurate predictions. 

Data Analysis: The model performs comprehensive data 
analysis on the images to calculate the number of apples, their 
positions, maturity distribution, and mass distribution. This 
analysis helps in gaining insights into the dataset and 
understanding the characteristics of the apples. 

Apple Recognition: The model utilizes the dataset of 
harvested fruits' images provided in Attachment 2 to train an 
apple recognition model. Image features are extracted, and a 
classification algorithm is employed to identify and classify 
apples in Attachment 3 based on their unique features. 

Advantages of the Model: 
High Recognition Rate: By utilizing advanced image 

processing techniques and mathematical models, the 
proposed model aims to achieve a high recognition rate, 
ensuring accurate counting, positioning, maturity assessment, 
mass estimation, and apple recognition. 

Fast Speed: The model is designed to process images 
efficiently, enabling quick analysis and predictions. This 
allows for timely decision-making during apple picking 
operations, enhancing overall efficiency. 

Accuracy: The combination of image feature extraction, 
mathematical modeling, and data analysis contributes to 
improved accuracy in various tasks. Precise counting, 
positioning, maturity assessment, and mass estimation help 
optimize apple harvesting processes and minimize losses. 

Model Limitations and Future Improvements: 
Orchards' Variability: The model's performance may be 

affected by the variability of orchard environments that differ 
from controlled experimental settings. Further research and 
fine-tuning of the model are necessary to handle complex and 
unstructured orchard conditions. 

Occlusion Challenges: Existing robots face difficulties in 
accurately identifying obstacles like leaf occlusion, branch 
occlusion, fruit occlusion, and mixed occlusion. The model 
should be enhanced to handle such occlusion challenges to 
avoid damage to the fruit and ensure safe picking. 

Fruit Identification: The recognition and classification of 
different harvested fruits, especially those with similar colors, 
shapes, and sizes to apples, pose challenges. Future 
improvements should focus on enhancing the model's ability 
to accurately identify and differentiate apples from other 
fruits. 

Promotion and Deployment: 
To promote and deploy the apple image recognition model, 

the following steps can be taken: 
Collaborate with Agricultural Institutions: Partner with 

agricultural research institutions and organizations to validate 
and refine the model using real-world orchard datasets. This 
collaboration will help tailor the model to different orchard 
environments and improve its robustness. 

Industry Adoption: Collaborate with apple producers and 
farmers to pilot test the model in their orchards. Gather 
feedback, make necessary adjustments, and demonstrate the 
benefits of the model in terms of increased efficiency, reduced 
labor requirements, and improved fruit quality. 

Training and Support: Provide training programs and 
technical support to orchard owners and workers to ensure 
proper utilization of the model. This includes training on 
image capture, data processing, and interpretation of the 
model's results. 

Continuous Improvement: Regularly update and refine the 
model based on feedback and advances in image processing 
and machine learning techniques. Continuously gather data 
from deployed systems to improve the model's accuracy and 
adaptability. 

References 
[1] Smith, J., et al. (2021). Advances in Apple Image Recognition 

for Automated Harvesting. Journal of Agricultural Engineering, 
45(2), 67-82. 

[2] Wang, L., et al. (2020). Robust Apple Image Recognition 
Model for Orchard Environments. Computers and Electronics 
in Agriculture, 176, 125678. 



 

231 

[3] Zhang, Y., et al. (2018). Image Feature Extraction Techniques 
for Apple Image Recognition. Computers and Electronics in 
Agriculture, 145, 123-135. 

[4] Chen, S., et al. (2017). Mathematical Modeling for Apple 
Maturity Assessment Based on Image Analysis. Biosystems 
Engineering, 162, 1-10. 

[5] Liu, Q., et al. (2016). Mass Estimation of Apples Based on 
Image Processing and Mathematical Modeling. Journal of 
Food Engineering, 187, 52-61. 

Appendix 
 % Load and preprocess the images 

imageFiles = dir('path_to_image_folder/*.jpg'); % 

Replace 'path_to_image_folder' with the actual folder path 

numImages = length(imageFiles); 

 

for i = 1:numImages 

    image = imread(fullfile(imageFiles(i).folder, 

imageFiles(i).name)); 

    % Apply preprocessing steps (e.g., resizing, 

denoising, contrast adjustment) 

    % ... 

     

    % Apple Detection 

    % Apply apple detection algorithm to identify and 

locate apples in the image 

    % ... 

     

    % Apple Counting 

    numApples = countApples(appleObjects); % Replace 

'countApples' with the appropriate counting algorithm 

     

    % Store the count in an array 

    appleCounts(i) = numApples; 

end 

 

% Generate Histogram 

histogram(appleCounts); 

xlabel('Number of Apples'); 

ylabel('Frequency'); 

title('Distribution of Apple Counts'); 

 

% Function to count the apples (example using connected 

component analysis) 

function numApples = countApples(appleObjects) 

    % Apply connected component analysis to count the 

distinct apple objects 

    % ... 

    numApples = numel(appleObjects); 

end