Closing the Loop in Industrial ML: Integration of Automated Density-Based Sampling in Production Pipelines

Data redundancy is one of the most relevant challenges in predictive maintenance applications based on industrial HVAC telemetry. It can increase computational costs and affect the performance and scalability of predictive models. This thesis studies Cluster-Based Density (CBD) sampling as a density-aware data reduction strategy to understand whether this approach can preserve predictive performance based on CatBoost models, embedded in an Anomaly Detection context. The work directly addresses the limitation of the density-based framework, which is manually configured and cannot be directly integrated into company production processes. The development of an automatic sampling pipeline allows this issue to be addressed. The empirical validation is conducted on Swegon air-to-water machines within the Statwolf industrial environment, examining both single-device and family level scenarios. CBD sampling is compared against size-matched simple random sampling (SRS). In the single device spatial experiments, CBD retains approximately 13.7%–14.7% of the original training data. Despite this reduction, CBD-based models achieve lower Mean Squared Error (MSE) than SRS across all target variables and operating regimes. The results remain favorable when the analysis is shifted to the entire family, although the advantage becomes more heterogeneous. The thesis also analyzes the temporal non-stationarity through zero-shot inference, static cumulative retraining, and adaptive retraining policies, also considering projection versus re-optimization strategies. Seasonal transitions affect both models trained on the entire data and those trained on the data reduced via CBD strategy, and, even if static retraining may prove beneficial, the results indicate that the high noise ingestion during the seasonal transition can lead to a degradation of predictive performance. Adaptive retraining strategies are therefore explored. Among the tested strategies, the 15–15 policy preserves the predictive performance of CatBoost models, and is generally more stable than smaller windows, such as the 7–7, especially during seasonal transitions. Projection and re-optimization are treated as complementary decisions, driven by the representativeness of the baseline and the fraction of noise generated when assigning new points to the clusters generated from the baseline. Finally, this work develops an automatic sampling pipeline designed for the integration of the CBD procedure into the Statwolf production workflow. This architecture supports different sampling options and exploits Bayesian optimization to automatically identify HDBSCAN hyperparameters that generate structurally coherent clusters, compared to the results obtained from manual tuning. The pipeline enables scalable projection by performing batch assignment of points excluded from the HDBSCAN fit. This contribution enables the CBD procedure to be transformed into a production-oriented component for automated data reduction. Overall, the results provide evidence that CBD sampling represents a valuable alternative to full-data training in the context considered. However, the entire process should be supported by human expert validation and temporal adaptation.