BookFusion a web-based book genre classification through multimodal ensemble of CNN and Vision Transformers - Dela Peña, Christian Carlo R.
Book genre classification is necessary for organization and accessibility of literature This serves various stakeholders including authors, readers, publishers, and libraries among others. This study presents an alternative approach to book genre classifi cation through the multimodal ensemble of convolutional neural networks (CNN), Vision Transformers (VIT), and transformers. By utilizing 57.000 book cover images and titles from BookCover 30 dataset, we explore the efficacy of ensembling differ-ent model architecture in enhancing the genre classification accuracy. The proposed ensemble model refine classification outcomes by aggregating predictions from dif ferent classifiers. The methodology begins with data preprocessing, model training and evaluation, culminating in the deployment of a web application with the best configuration of the model. Experimental results indicate significant performance improvement in terms of accuracy and efficiency trade-off, better than state-of-the-art methods. Incorporating text inputs noticeably contributes to better classification performance. This research not only contributes to the field of literature organization but also sets the groundwork for future explorations into ensemble and multimodal learning frameworks.