21th AIAI 2025, 26 - 29 June 2025, Limassol, Cyprus

Automating Industrial Quality Control: A Multimodal LLM and RAG Framework for Anomaly Detection

Bianchini Filippo, Calamo Marco, Marinacci Matteo, Rossi Jacopo, Mecella Massimo

Abstract:

  Anomaly detection in many industrial productions, including electronic boards, is a critical aspect of quality control, demanding high precision and reliability. Traditional inspection workflows rely heavily on manual expertise, making them labor-intensive, error-prone, and difficult to scale. In this paper, we present an open-source novel framework that integrates Large Language Models (LLMs) with Retrieval-Augmented Generation (RAG) and multimodal analysis to enhance anomaly detection. Our approach consists of two core components: (i) an AI-powered assistant capable of extracting and interpreting information from structured and unstructured documentation, and (ii) a multimodal LLM-based system that fuses textual and visual data to identify defects in electronic board images. To evaluate the framework, we conduct an extensive assessment using three open-source LLMs, benchmarking their performance on IPC-A-610F, the industry standard for electronic board defect detection. Key metrics, including accuracy, faithfulness, and retrieval precision, are analyzed to determine the system’s effectiveness. Results indicate that LLMs significantly enhance information retrieval and defect detection, providing a scalable, efficient solution for industrial applications.  

*** Title, author list and abstract as submitted during Camera-Ready version delivery. Small changes that may have occurred during processing by Springer may not appear in this window.