Robotics & Autonomous Systems

Robot Hardware & Vision System

The robots utilise an OmniDirectional Vision System positioned at the highest permissible point to maximise field of view. This system integrates three cameras, each phased by 120° and tilted downwards at 30°, providing a 360° view without blind spots. Each camera (OV2710 sensor) operates at 640 × 480 pixels with a 125° FOV at 120 fps, ensuring high frame rates. An Adafruit BNO085 digital compass provides yaw values for orientation, complementing the visual data.

Developed Solution Architecture

The proposed system leverages neural networks to interpret visual data and precisely determine robot positions. It's particularly effective in dynamic environments like football. The solution consists of four main components:

The system detects FIFA and MSL-regulated field markings (e.g., corner marks, penalty spots, center circle, 'L', 'T', '+' shapes, goalposts) without requiring field modifications. Initial development used Webots R2023a simulation for data generation, followed by manual annotation and Roboflow data augmentation (flip, rotation, crop, saturation, brightness, exposure, blur, noise) to create a robust dataset of simulated and real images.

Neural Network Training & Results

The Marker Detection Neural Network uses YOLOv10s, chosen for its efficiency and accuracy in detecting seven unique marker types. Training involved 1500 epochs with automated batch sizing and 640 × 640 pixel input images. The model achieved a mean Average Precision (mAP) of 0.896 at an IoU threshold of 0.5, demonstrating high detection accuracy despite white markers on a green background. Most markers showed true positive rates above 85%.

For Camera-World Coordinates Transformation, a dedicated regression neural network maps screen coordinates (u, v) to real-world angle and distance. This model, trained on 545 data points, eliminates the need for explicit formulas and adapts to lens variations. To ensure real-time performance, pre-calculated coordinates are integrated into a lookup table, requiring 14.74 MB of memory for 640x480 resolution.

The Localisation Neural Network is a feedforward regression network with 31 inputs (angle and distance from detected markers, plus compass orientation). It was trained on 40,000 data points across the football field, with noise (±10 degrees for angle, ±20 cm for distance) and marker normalisation (0-1 range) to enhance resilience. Closer markers were prioritised. The X-axis network achieved an average error of 3.82 cm, with 80.78% accuracy within 20 cm. The Y-axis network achieved an average error of 1.61 cm, with 91.55% accuracy within 20 cm.

Limitations & Advantages

The system has a detection limit of 6 meters for most markers (10 meters for goalposts) due to camera resolution (640 × 480 pixels). This can compromise localisation in distant field zones. Dynamic game conditions, where markers might be obscured by other robots or objects, also pose a challenge to the system's robustness.

A significant advantage is its independence from colour transitions, making it resilient to lighting variations. The auto-calibration of cameras and neural network adaptability further enhance robust marker detection. Data augmentation during training improves generalisation across diverse environmental settings.

Discussion and Future Directions

Marker detection accuracy significantly impacts overall localisation performance; fewer detected markers can lead to increased errors. The system addresses data redundancy from overlapping camera views by processing each marker detection once. Prioritising closer markers enhances learning and robust predictions, especially with varying marker densities.

Simulation with Webots and data augmentation with Roboflow proved invaluable for creating diverse, realistic datasets. While robust in typical scenarios (errors within 90 cm less than 0.43% of occurrences), extreme conditions with limited or obstructed markers remain a challenge. The 50 cm robot diameter makes observed errors negligible in most gameplay.

Sensor fusion, integrating locomotion encoders and compass data, enhances stability by mitigating positional jumps. A Kalman filter refines these estimates, ensuring smooth and consistent localisation. The system processes both classification and regression networks in an average of 38 ms (26.3 Hz), slightly faster than the communication system (25 Hz), with potential for further optimisation.

Conclusions

This study successfully introduced a novel multi-camera localisation system for robotic football, integrating classification and regression neural networks. The classification network detects markers, which are then processed by two regression networks to compute precise (x, y) coordinates. This architecture effectively bridges traditional vision-based methods with modern AI, offering adaptability to complex spatial relationships and dynamic environments.

The system demonstrated robust, scalable, and efficient real-time performance through computational optimisations like pre-computed transformations. Its success is attributed to a diverse, high-quality marker dataset, despite visibility remaining a critical factor. Validated in the RoboCup MSL, the system achieved acceptable error margins, earning a first-place award for its innovation, efficiency, and adaptability.