African Journal of Food Science and Technology

Application of Convolutional Neural Network (CNN) on the Post-harvest Changes of Some Selected Climacteric Fruits

George Ifeoluwa Pele^1* and Simeon Adedoyin Adeyemi^{2 1}Department of Food Technology, Federal Polytechnic, P.M.B. 420, Offa, Kwara State, Nigeria
²Department of Computer Science, Federal Polytechnic, P.M.B. 420, Offa, Kwara State, Nigeria
^*Corresponding Author:
George Ifeoluwa Pele, Department of Food Technology, Federal Polytechnic, Nigeria, Tel: +2348034899527, Email: ife_pele@ymail.com

Received: 11-Nov-2023, Manuscript No. AJFST-23-119881; Editor assigned: 13-Nov-2023, Pre QC No. AJFST-23-119881; Reviewed: 27-Nov-2023, QC No. AJFST-23-119881; Revised: 29-Nov-2023, Manuscript No. AJFST-23-119881; Published: 11-Dec-2023, DOI: 10.14303//ajfst.2023.051

Introduction

Fruits have long been a significant component of the human diet, contributing diverse quality attributes such as colors, shapes, flavors, aromas, and textures. While their nutritional importance has gained recognition with the increasing awareness of food quality and safety (Adeeko et al., 2020), the consumption of fruits has been linked to various health benefits, including a reduced risk of cancer and heart diseases (Paulauskiene et al., 2020). Climacteric fruits, regulated by ethylene production, continue to ripen post-harvest due to a surge in respiration during ripening onset (Dallenbach et al., 2020). Despite their pivotal role in human nutrition, the processes of maturity, ripeness, and senescence in fruits remain intricate, controlled by multiple developmental and environmental signals with molecular mechanisms that are not yet fully understood (Shimshoni et al., 2019). Ripening involves crucial metabolic changes such as chlorophyll breakdown, pigment accumulation, cell wall degradation, and the synthesis of low-weight metabolites, enhancing the appeal of fruits to seed dispersers (Mpai et al., 2020).

In recent years, Convolutional Neural Networks (CNNs) have gained prominence, particularly in image processing, inspired by the organization of the cat's visual cortex. CNNs impose local connectivity on raw data, perceiving images as collections of local pixel patches rather than independent pixels. This approach extracts more significant features and reduces the number of parameters by using filters or kernels to calculate the weights of input (Zhang et al., 2019).

Apple (Malus domestica) contains biologically active compounds and phenolic compounds with antioxidant properties, making it a year-round staple consumed fresh or processed into juice and other products (Dita et al., 2007). Bananas, a vital and economical fruit globally, experiences rapid physiological deterioration post-harvest and play a crucial socio-economic role in tropical and sub-tropical regions (FAO, 2019). Guava, known for its antioxidants and phytochemicals, including ascorbic acid, carotenoids, dietary fiber, and polyphenolics, ranks second only to acerola cherries in ascorbic acid content (Amri, 2010).

The study's objective was to develop a predictive model for post-harvest changes in selected climacteric fruits using Convolutional Neural Network (CNN) technology.

Materials and Methods

Sample collection

Matured climacteric fruits, specifically Apple, Banana, and Guava, were sourced from Erefe Farm Settlement, Ile-Ife, Nigeria, for an immediate quality study.

Image data collection

Digital images of distinct sections of the fruits were captured in JPEG format using a digital camera. Simultaneously, selected fruits kept at room temperature were subjected to evaluation for physicochemical properties, proximate, and vitamin compositions, following the guidelines outlined by AOAC (2005).

Mapping of images with quality attributes

A predictive model, utilizing the Convolutional Neural Network (CNN) algorithm, was developed to map images of the chosen fruits to their respective identified quality attributes. Prior to processing, image data underwent preprocessing to enhance relevant features such as color, size, and orientation using the CNN algorithm. The mapping of images and the analysis of quality attributes were carried out employing the Convolutional Neural Network (CNN) algorithm. The resultant predictive model was implemented as a mobile application using the Python programming language. To assess the model's performance, Accuracy, Precision, and Recall were employed as evaluation metrics.

Analyses

Proximate analysis

The physicochemical, proximate, and vitamin analyses were conducted following the procedures outlined by the Association of Official Analytical Chemists (AOAC, 2005).

Data collection

The results of the physicochemical properties, proximate, and vitamin compositions are detailed in (Tables 1-9). Images of each selected fruit were systematically captured at different postharvest intervals. This approach aimed to encompass variations in natural environments, facilitating model generalization during subsequent inferencing stages. The images were taken at diverse angles, orientations, under varying lighting conditions, and at different times of the day. This diverse set of images served as the dataset for both training and evaluating the Convolutional Neural Network (CNN)-based models. The dataset comprises a total of 599 Apple images, 779 Banana images, 400 Cucumber images, and 639 Guava images. The intention behind assembling this dataset was to construct two distinct classifiers: one for identifying the type of fruit and the other for determining the specific day of the fruit, enabling estimation of its properties. Subsequently, splits of each fruit type categorized into their respective days were created, constituting the third category of datasets derived from the second image dataset.

Data preprocessing

Data preprocessing became imperative due to the inherent challenge of achieving absolute capture of fruit images. The initial preprocessing step involved cropping the images around the fruit, accomplished using a free online tool accessible at https://bulkimagecrop.com/

To ensure uniformity, all images were resized to the same dimensions using a freely available online tool supporting bulk image resizing, found at https://bulkresizephotos. com/en. Specifically, all cropped images were resized to 256x256, a dimension often recommended as optimum for deep learning projects in scientific research. These processed images constituted the dataset for both training and evaluating the CNN-based models.

Image segmentation

Following the cropping and resizing of images to a uniform dimension, the subsequent step involved splitting the images into training, validation, and testing sets. Adhering to a conventional split ratio, the images were divided into 70%, 20%, and 10% for training, validation, and testing sets, respectively. This same split ratio was applied to prepare datasets for both training and evaluating both the fruit classifier and the fruit-day classifier, as outlined in (Table 4).

Image augmentation

Recognizing that Convolutional Neural Network (CNN)- based algorithms benefit from a larger volume of training images to capture real-life variations, image augmentation was deemed necessary. Additional training images with slight variations and deformations of existing training images were generated using the Image Data Generator class, a preprocessing class for image augmentation available in the Keras Python framework. This augmentation process enhanced the diversity of the training dataset, contributing to the robustness of the CNN-based models.

The augmentation process involved generating additional training images through various transformations. Initially, the images were rescaled by a factor of 1/255. Subsequently, random deformations were applied to sampled images to create a more extensive set of training images. These deformations included zooming in by 10% (0.1), modifying shear within a range of 0.2, performing random horizontal flips, rotating images by 10 degrees, and shifting image width within a 10% range of the original width and original height. Additionally, the augmentation process incorporated mean subtraction, a scaling/normalization technique proven to enhance model accuracy.

This comprehensive augmentation strategy aimed to introduce diversity and variability to the training dataset, enabling the Convolutional Neural Network (CNN)-based models to better generalize and capture the complexities associated with real-world data.

Model training

Model training was conducted on Google Colab, a cloudbased tool offering fast GPU processing and extended RAM memory, essential for the computational demands of deep learning projects. Seven CNN-based models were trained: one for classifying images into their respective types, and six specific to each fruit type, classifying fruits into their respective days after harvest. The resulting day predictions were then used to estimate various fruit properties.

Due to the limitation of data for single fruit analysis, a regression model for understanding property value patterns could not be implemented. Instead, the models could only estimate properties based on the predicted day from the respective day-classifier model. The CNN architecture in (Figures 1 and 2) was utilized for all models, with variations in the number of nodes in the output layers. The fruitclassifier model had 3 output nodes, while the Fruit-days classifier models for Apple, Banana, and Guava had 8, 9, and 7 output nodes, respectively. All convolutional layers produced 32 filters and utilized max-pooling with a stride of 2 x 2. Five convolutional layers were employed in both architectures, with the first connected directly to the input layer. Subsequent convolutional layers were followed by max-pooling layers to focus on extracting prominent features from each filter. This approach reduced the number of nodes before flattening for the fully-connected neural network layers, reducing computational complexity, memory requirements, and model size.

The models were trained using the "Adam" optimizer, an adaptive form of Stochastic Gradient Descent (SGD) optimization, addressing the vanishing gradient problem with categorical-cross entropy loss measure. To expedite convergence and approach the global minimum of the loss function, a learning rate (LR) annealing method was applied, gradually reducing the LR by half (0.5) when the validation accuracy showed no improvement over three consecutive steps. The architectures depicted in (Figures 1 and 2) consisted of the following layers: [Provide details on the layers as shown in the figures.

Input: [Conv2d(32 x (256 x 256 x 3)) + ReLU]

Conv1: [Conv2d(32 x (256 x 256 x 3)) + ReLU + MaxPool2D((2,2))]

Conv2: [Conv2d(32 x (256 x 256 x 3)) + ReLU + MaxPool2D((2,2))]

Conv3: [Conv2d(32 x (256 x 256 x 3)) + ReLU + MaxPool2D((2,2))]

Conv4: [Conv2d(32 x (256 x 256 x 3)) + ReLU + MaxPool2D((2,2))]

Hidden1: [FC(128) + ReLU]

Hidden1: [FC(32) + ReLU]

Output: [FC(6) + Softmax]

Introduced to dynamically adjust the learning rate (LR) based on the validation accuracy, this annealing mechanism allows the model to navigate the loss landscape more efficiently, preventing convergence issues associated with a fixed, high learning rate. To manage computational resources effectively and mitigate overfitting during training, an early-stopping mechanism was implemented. Training would cease if the validation loss did not decrease after five consecutive rounds, indicating that the model had likely reached its learning plateau and further training might lead to overfitting. Each model was fitted to the training set images and validated using the respective validation sets for 100 epochs. A batch size of 128 was employed for the fruit-classifier, while a batch size of 16 was used for the six fruit-day classifiers. The smaller batch size for the fruit-day classifiers was due to the limited number of images available for some day categories in the training and validation sets. It is important to note that, owing to the early-stopping mechanism; most models did not train for the full 100 epochs but stopped earlier when the best weights were returned. The training reports for each model are detailed in (Figures 3 to 6). (Figure 3) shows fluctuations in the fruit classifier model up to epoch 22, after which the learning process stabilized. Investigation revealed that this stabilization coincided with the introduction of the "ReduceLROnPlateau" annealer mechanism, indicating its positive impact on the training process.

The normalization observed in the learning process of the fruit classifier model at epoch 22 was a result of the application of the "ReduceLROnPlateau" annealer mechanism. This mechanism reduced the learning rate (LR) by half, from the initial value of 0.001 to 0.0005. Subsequent reductions occurred at epoch 33, further lowering the LR to 0.00025. The learning process stopped at epoch 44 due to the "Early Stopper" mechanism, which was also in place to prevent overfitting. For the Apple Fruit-Day classifier training (Figure 3), the observed normalization was also influenced by LR reductions at epochs 12 and 21. The training process stopped at epoch 63 due to the "Early Stopper" mechanism. In the case of the Banana Fruit-Day classifier, LR reductions occurred at epochs 12 and 24, with the training process concluding after 53 epochs. The Guava Fruit-Day classifier training (Figure 5) exhibited moderation starting at epoch 15, and there was no significant improvement in learning thereafter.

The training process stopped at this point. These observations highlight the effectiveness of the annealing mechanism in dynamically adjusting the LR, allowing for more efficient convergence and preventing potential issues associated with a fixed, high learning rate. The "Early Stopper" mechanism further contributed to optimizing the training process by preventing overfitting and conserving computational resources.

Model evaluations

The trained models were evaluated on the testing set datasets using accuracy, precision, recall, ROC-AUC, and F1- measure classification metrics. Given a sample Confusion Matrix as shown in (Table 5), these metrics are shown in equations (1) to (5) below:

Accuracy: Ratio of correct prediction to all predictions. It is interpreted as the measure of the overall performance of a model, defined by Eqn 3.6 below. accuracy=

(TP+TN)/((TP+TN+FP+FN)) (1)

Recall (Sensitivity): proportion of positive data points that are correctly considered as positive, with respect to all positive data points. It is a measure of the coverage of actual positive samples, defined as shown in Eqn 3.7 below.recall=

TP/((TP +FN)) (2)

Precision: Ratio of correctly predicted positives to all positive predictions. It is a measure of how accurate the positive predictions are, defined in Eqn 3.8 below. precision=

TP/((TP + FP)) (3)

F1: Harmonic mean (average) of precision and recall. Maintains balance between precision and recall. A useful metric for unbalanced classes, calculated as shown in Eqn

3.9. f1= 2 x precision x recall/(precision + recall) (4)

ROC-AUC: ROC AUC stands for Receiver Operating Characteristic Area Under the Curve. The ROC-AUC score measures a model's performance at distinguishing between the positive and negative classes ranging between 0 and 1, with a score of 1 implying a classifier can perfectly distinguish between all the positive and all the negative class points, while a score of 0.5 implies that the classifier is unable to make distinction between the two classes.

roc_auc= (TP x FP)/2+TP x (1-FP)+((1-TP)x (1-FP))/2 =(1 + TP - FP)/2 (5)

Model deployment and inferencing applications development

The trained models were firstly deployed to a local server using Streamlit Python framework during development and then deployed in production to Python-Anywhere. Two (2) Inference applications were developed, one for the web channel and the other for mobile channel. The web channel was implemented using Flask, HTML, CSS and JavaScript. Flask was used for backend part of the web application accepting prediction requests and serving the prediction results to requesting client Applications. The frontend web application was developed using HTML, CSS and JavaScript. The mobile application part was implemented using Flutter framework and Dart programming language. The same backend was used to serve both the web frontend and the mobile applications.

Results and Discussion

Predictive analysis

The results (Figure 7) showed that all the 61 apple fruits and 65 Guava fruits were correctly classified. Out of the 79 Banana fruit images evaluated, there were 77 correct classifications with 2 misclassified as Guava. By this result, the trained model was able to achieve 100%, 97.5% and 100% predictive accuracies for Apple, Banana, and Guava, respectively. As shown in (Figure 8), the Apple-Days Classifier model correctly predicted all 61apple images testing set into their respective days with 100% accuracy. Except for 1 Banana belonging to Day11 which was misclassified as Day2, all other Bananas were correctly classified into their respective days by the Banana-Days Classifier, as shown in (Figure 9). All 71 guava images used for testing were correctly classified into their respective days by the trained Guava-Days classifier model, demonstrating a 100% predictive accuracy, as shown in (Figure 10). The performance evaluation for all the seven (4) trained models are shown in (Table 5), As shown in the results (Table 5), the models for classifying Apple and Guava days had the overall best performances. Judging by the consistently high ROC-AUC score across all trained models, it can be seen that all the trained models are highly capable of accurately distinguishing one class from another (Table 9)The snapshots of the hosted Web-based Inference Application and Mobile Inference Application for end-users are shown in (Figure 12 to 15). Though Sakib et al. (2019) worked on the implementation of fruits recognition classifier using Convolutional Neural Network algorithm for observation of accuracies for various hidden layers where a flood full type algorithm was developed which extract the fruit from background where all the fruits were scaled down to 100 x 100 pixels of standard but the inadequacy of the work is that the computational model used could not predict the quality attributes inherent in the fruit. Pathak, (2021) also studied the classification of fruits using Convolutional Neural Network and Transfer Learning Model where a deep learning based model for classification of fruit freshness implemented by using public data set named as "fruit fresh and rotten for classification" derived from kaggle was experimented, however the study is only limited to the classification of the fruits while the quality attribute inherent in the fruit was not determined (Table 10, 11)

Conclusion

The present study has demonstrated the development of a predictive model for postharvest changes of climacteric fruits where the trained model was able to achieve 100%, 97.5%, 100% predictive accuracies for Apple, Banana, Guava, respectively. The predictive model has shown that it has the capability to enhance the consumers’ predictive insight towards the quality attributes inherent in some selected climacteric fruits at a real time before money.

References

Adeeko A, Yudelevich F, Raphael G, Avraham L, Alon H, et al. (2020). Quality and storability of trellised greenhouse-grown, winter-harvested, new sweet acorn squash hybrids. Agronomy. 10: 1443.

Indexed at, Google Scholar, Cross Ref

Amri E (2010). Germination of terminalia sericea buch ex dc. seeds: Effects of temperature regime, photoperiod, gibberellic acid and potassium nitrate. Am Eurasian J Agric Environ Sci. 8: 722-727.

Indexed at, Google Scholar

AOAC (2005).Association of Analytical Chemist, Official Methods of Analysis. In W. Horowitz 18thed. AOAC, Gaithersburg, MD

Dallenbach LJ, Eppler T, Buhlmann-Schutz S, Kellerhals M, Buhlmann A (2020). Pre-and postharvest factors control the disease incidence of superficial scald in the new fire blight tolerant apple variety “Ladina”. Agronomy. 10: 464.

Indexed at, Google Scholar, Cross Ref

Dita M, Barquero M, Heck D, Mizubuti ES, Staver CP (2018). Fusarium wilt of banana: Current knowledge on epidemiology and research needs toward sustainable disease management. Front Plant Sci. 9: 1468.

Indexed at, Google Scholar, Cross Ref

FAO (Food and Agriculture Organization of the United Nations) (2019). FAOSTAT agricultural production.

Mpai S, Sivakumar D (2020). Stimulation of light-emitting diode treatment on defence system and changes in mesocarp metabolites of avocados cultivars (Hass and Fuerte) during simulated market shelf conditions. Agronomy. 10: 1654.

Indexed at, Google Scholar, Cross Ref

Pathak R, Makwana H (2021). Classification of fruits using convolutional neural network and transfer learning models. J Manag Inf Decis Sci. 24: 1-12.

Indexed at, Google Scholar, Cross Ref 

Paulauskiene A, Taraseviciene Z, Zebrauskiene A, Pranckietiene I (2020). Effect of controlled atmosphere storage conditions on the chemical composition of super hardy kiwifruit. Agronomy. 10: 822.

Indexed at, Google Scholar, Cross Ref

Sakib S, Ashrafi Z, Siddique MA (2019). Implementation of fruits recognition classifier using convolutional neural network algorithm for observation of accuracies for various hidden layers. Appl Sci. 1904:00783.

Indexed at, Google Scholar

Shimshoni JA, Bommuraj V, Chena Y, Sperling R, Barel S, et al (2019). Residual distribution kinetics of pesticides in cherry tomato peel, pulp, and fruit as a function of irrigation water salinity, household rinsing, and storage regimen. Agronomy. 9: 800.

Indexed at, Google Scholar

Zhang YD, Dong Z, Chen X, Jia W, Du S, et al (2019). Image based fruit category classification by 13-layer deep convolutional neural network and data augmentation. Multimed Tools Appl. 78: 3613-3632.

Indexed at, Google Scholar, Cross Ref