A Performance Comparison of Machine Learning Models for Robotic Navigation Using Imbalanced and SMOTE-Enhanced Data

Abstract

Robotic navigation systems rely heavily on accurate classification of sensor inputs to make real-time movement decisions. However, class imbalance—where certain navigation commands are underrepresented—can severely degrade the performance of machine learning models, particularly in safety-critical scenarios. This study presents a comprehensive comparative analysis of six supervised learning classifiers—Decision Tree (DT), Random Forest (RF), k-Nearest Neighbors (KNN), Ranger, Support Vector Machine (SVM), and XGBoost—applied to the UCI Wall-Following Robot Navigation dataset. The objective is to evaluate classifier performance across both the original imbalanced dataset and a SMOTE-balanced version, and to understand the impact of data balancing on classification accuracy. A range of performance metrics was employed, including Accuracy, F1 Score, Precision, Recall, Specificity and Area Under the Curve (AUC). Results show that ensemble-based classifiers, particularly XGBoost and Ranger, significantly outperform traditional models under both data conditions, achieving near-perfect performance with F1-scores above 0.995 and AUC values of 1.000. In contrast, KNN and SVM showed limited robustness to class imbalance, with substantial drops in F1 and AUC when exposed to synthetic samples generated by SMOTE. The study highlights that while SMOTE improves minority class recall across most models, it may introduce noise and overlapping classes, especially detrimental to distance- and margin-based classifiers. Overall, the findings emphasize the effectiveness and resilience of ensemble models in managing imbalanced, multi-class robotics data. The study concludes that both classifier selection and data preprocessing are critical for developing reliable, real-time robot navigation systems. Recommendations include using XGBoost or Ranger in such applications and applying SMOTE selectively based on model characteristics