MarketNet multicolor hybrid CNN and ViT for multitask image classification & segmentation - Phillips, Charles Roy R. Jr
Monitoring grocery inventory is important for effective supply chain management and reduction of waste. Current grocery image classification models generally rely on RGB images but fail to capture the complex variations of grocery items. This paper aims to counter that limitation by introducing the novel deep learning model, known as MarketNet, which specifically aims at grocery image-based classification and segmentation tasks using multi-input channels. MarketNet uses a multi-input convolutional neural network (CNN) integrated with a Vision Transformer (VIT) to improve classification and segmentation accuracy The model incorporates multicolor input to learn richer and more discriminative features. The method was tested using a grocery image dataset in which the model classified items into predefined categories while simultaneously segmenting images for further localization. The methodology also includes preprocessing grocery images into multiple color channels followed by feature extraction by CNN. Features are further improved by the Vision Transformer to better accuracy and efficiency in predicting. It is shown experimentally that MarketNet does better in comparison to its baseline variants with the following metrics 98.87% Top-1 accuracy and 99.93% Top-5 accuracy for classification, and 94.54% mIoU, 96.73% accuracy, 96.81% precision, 97.54% recall, and 97.19% F1-Score far segmentation. This work contributes to the progress of grocery image analysis as a framework that can support improved inventory management, reduction in waste, and real-time product tracking within grocery settings.