End-to-End AI and Computer Vision Automation for Used Vehicle Image Analysis

This thesis introduces a comprehensive Artificial Intelligence (AI)-powered backend system specifically designed for Codebeex, a company dedicated to establishing a centralized online platform for used car and reusable car part dealerships. The core objective of this project is to automate the visual analysis of vehicle images by deploying a suite of modular, production ready Application Programming Interface (API) services that take advantage of advanced computer vision techniques. The developed system integrates several key functionalities: • Image Classification and Validation: Uploaded vehicle photographs are automatically categorized into five predefined angle-based classes. The system rigorously validates image suitability based on viewpoint, ensuring only images of the vehicle’s exterior proceed for further processing. • Precise Main Vehicle Segmentation and Background Removal: Employing a synergistic approach combining You Only Look Once (YOLO)11 for accurate vehicle detection and the Segment Anything Model (SAM) for fine-grained segmentation, the system isolates the primary vehicle. This process effectively eliminates background clutter, such as other vehicles, people, and environmental distractions, producing a clean, white background image that significantly enhances the performance of subsequent analytical models. • Intelligent Privacy-Preserving Text and Logo Blurring: For selected images, text and logo regions are automatically identified utilizing Florence-2 OCR and a specialized YOLO11-based logo detection model. A unique aspect of this system is its selective blurring mechanism: only non-original elements like dealer decals, aftermarket logos, or user-added text (e.g. slogans, website links, contact information) are blurred, while official manufacturer logos and brand names (e.g., Fiat, Ford) are intentionally preserved to maintain vehicle identity. • Automated Car Part and Damage Analysis for Reusability Assessment: The system incorporates two distinct segmentation-based YOLO11 models. The first identifies individual exterior car components (e.g., hood, windshield). Subsequently, a dedicated damage segmentation model detects visible surface damages. The spatial intersection of these outputs allows for precise identification of damaged car parts. If a damage region significantly overlaps a car part, that part is classified as damaged; otherwise, it is accepted undamaged and potentially reusable. A final summary report is generated, listing all identified vehicle parts and their condition to support decision-making for repair, resale, or recycling. The performance of each integrated model was thoroughly evaluated using task-specific metrics. For classification, accuracy, confusion matrices, and inference time were reported. Segmentation models were assessed via Mean Average Precision (mAP) and True Positive Rate (TPR). Comprehensive inference time statistics, including average, minimum, maximum, and standard deviation, were meticulously collected across various Fast Application Programming Interface (FastAPI) endpoints, unequivocally demonstrating the system’s computational efficiency, scalability, and readiness for real-world deployment. This research showcases the practical integration of cutting-edge AI and computer vision techniques, providing a scalable and automation-driven solution for the e-commerce domain of used vehicles.