1 of 16

Project Title: Robotic Bird Deterrent

University: WPI

Principal Investigator(s): G. Lewin, J. Xiao

Team Members: Kunal Nandanwar

Funding Amount/Source: $35k

Schedule:

Deliverables:�

Robot design�
Algorithm/implementation for detecting birds�
Results of the system evaluation

Objectives:�

Develop effective methods for deterring ravens from damaging substations

Develop efficient methods for detecting and identifying birds

Problem statement:�

Create automated systems for deterring birds from congregating near utility assets

Evaluate methods for dispelling birds

PROJECT SUMMARY

research deploy train evaluate

Hello everyone, my name is Kunal Nandanwar and I'm here to present to you our project summary on the "Robotic Bird Deterrent" at WPI. Our project is focused on addressing the problem of birds congregating near utility assets and causing damage.

we aim to create effective methods to deter birds from damaging substations and identifying birds efficiently. Birds, especially ravens, can cause serious damage to utility assets, resulting in costly repairs and outages. Therefore, it is essential to find a way to keep birds away from these assets.

Our objectives are to develop effective and efficient methods to deter ravens and identify birds using automated systems. This would involve developing a robot design and an algorithm for detecting birds. We plan to test and evaluate our system to ensure its effectiveness and efficiency.

2 of 16

Last Review Status:

A basic prototype of a power line patrolling robot was developed with radio control and custom phone app options, as well as the ability to operate autonomously.

System testing was conducted to ensure the robot's robustness before testing it on a power line.

Designed robot's detection and deterrents to enhance its effectiveness

A charging station design was in progress, with the robot being capable of connecting to a charger and recharging itself, though reliability testing was still in its early stages.

Eversource identified an electrical substation where ravens caused damage, presenting a less risky opportunity to deploy and test the system.

PROGRESS SUMMARY

research deploy train evaluate

During our previous meeting, we made significant progress towards developing a basic prototype of the robot, which included both radio control and custom phone app options, as well as autonomous capabilities.

In order to ensure that the robot was robust enough to handle the demanding environment of power lines, we conducted extensive system testing. We also designed the robot's detection and deterrents to enhance its effectiveness in preventing damage caused by ravens.

We were in the process of designing a charging station for the robot and the reliability testing of the charging station was in its early stages.

In addition to these developments, Eversource identified an electrical substation where ravens have caused significant damage. This presented us an opportunity for us to deploy and test our system in a less risky environment. We are currently in discussions with Eversource to finalize the deployment plans and hope to share the results of the testing with you soon.

3 of 16

PROGRESS SUMMARY

Inflatable deterrent and dead raven deterrent

research deploy train evaluate

4 of 16

Progress since Last Review:

The robot can detect ravens using computer vision and onboard webcam along the transmission line

Visual and audio stimuli are used to deter the ravens located collinearly along the line

Information is relayed to the mobile app in real-time while the robot drives along the transmission line

The robot can be charged wirelessly using the Qi-wireless pad located on the bottom of the battery compartment

Vision algorithms were trained on stock images and open-source datasets of ravens, received the test accuracy of ~98%

PROGRESS SUMMARY

research deploy train evaluate

Our robot is now equipped with computer vision and an onboard webcam, which allows it to detect ravens along the transmission line. Once detected, visual and audio stimuli are used to deter the ravens located collinearly along the line. We are also able to relay information to the mobile app in real-time while the robot drives along the transmission line.

We have also implemented a wireless charging feature using a Qi-wireless pad located on the bottom of the battery compartment. This makes charging the robot much more convenient and efficient.

Furthermore, we have trained our vision algorithms on stock images and open-source datasets of ravens, achieving an impressive test accuracy of around 98%. This has significantly improved the robot's ability to detect ravens accurately and efficiently.

5 of 16

TECHNICAL SUMMARY

research deploy train evaluate

6 of 16

YOLOv5 (You Only Look Once version 5): Object detection algorithm that uses a deep neural network to detect objects within an image

YOLOv5 trained on a dataset of images that contain ravens, as well as images that do not contain ravens

The model learns to identify the unique features of a raven, such as its shape, size, and color, and use this information to detect ravens within new images

YOLOv5 predicts a confidence score for each bounding box, which indicates how likely it is that the box contains an object

The model predicts a class label for each bounding box, which specifies what type of object the box contains (in this case, a raven)

TECHNICAL SUMMARY

research deploy train evaluate

7 of 16

Dataset: Caltech-UCSD Birds, OIDv4, NABirds, and custom images of ravens

The Caltech-UCSD Birds dataset includes images of birds in natural settings, while the NABirds dataset focuses on birds found in North America

The OIDv4 dataset is a large-scale object detection dataset that includes images of various objects, including birds

The custom images of ravens were used to create a diverse dataset for training the AI model

TECHNICAL SUMMARY

research deploy train evaluate

8 of 16

Data augmentation involves applying various transformations to existing data, such as changing illumination and adding noise or blurring, to generate new and diverse training data

This technique is crucial for increasing dataset size and improving the accuracy of trained models

In our case, data augmentation was essential to create a more comprehensive and realistic dataset of ravens, with different features such as size, color, and posture, to enhance the accuracy of the vision algorithms in detecting them along power lines

TECHNICAL SUMMARY

research deploy train evaluate

9 of 16

TECHNICAL SUMMARY

research deploy train evaluate

This is the basic flowchart of how to train the model on YOLOv5:

Input Data: The first step is to provide the custom images and their respective annotations in a suitable format to the YOLOv5 training pipeline.
Data Preprocessing: The data is preprocessed by resizing the images and applying data augmentation techniques to improve the robustness of the model.
Train/Val: The data is split into training and validation sets.
Model Training: The YOLOv5 architecture is trained on the custom images and their annotations using the training set.
Model Evaluation: The performance of the model is evaluated using detection performance metrics such as mean average precision (mAP) on the validation set.
Fine-tuning: The model is fine-tuned by adjusting the hyperparameters and retraining it on the training set to further improve its performance.
Model Deployment: The trained model is deployed for inference, i.e., it is used to detect objects in new images.
Output: The output of the model is the detected objects and their respective bounding boxes in the new images.

10 of 16

TECHNICAL SUMMARY

research deploy train evaluate

11 of 16

TECHNICAL SUMMARY

research deploy train evaluate

12 of 16

TECHNICAL SUMMARY

research deploy train evaluate

13 of 16

TECHNICAL SUMMARY

research deploy train evaluate

14 of 16

The robot was designed with modularity in mind for easy modification of subsystems such as deterrents and camera modules

To combat raven adaptation, the robot utilizes both visual and auditory deterrents, including LED strobing lights and a series of speakers broadcasting a 4000 Hz signal noise

Laser Time of Flight (ToF) distance sensors were used for bird safety and to improve read distance, with a range of up to 2 meters and a smaller software library for implementation with the Arduino board

The primary detection and deterring methods of the robot are controlled by a state machine that interprets data from the camera and ToF sensors and enacts one to two possible states: patrolling and detecting

If the AI fails to detect a raven, the ToF sensors were calibrated to detect ravens based on proximity to the sensor(s), with a detection distance of 0.5 meters to allow for the robot to come to a complete stop before making contact with the raven

TECHNICAL SUMMARY

research deploy train evaluate

The robot was designed with modularity in mind for easy modification of subsystems such as deterrents and camera modules

To combat raven adaptation, the robot utilizes both visual and auditory deterrents, including LED strobing lights and a series of speakers broadcasting a 4000 Hz signal noise

Laser Time of Flight (ToF) distance sensors were used for bird safety and to improve read distance, with a range of up to 2 meters and a smaller software library for implementation with the Arduino board

The primary detection and deterring methods of the robot are controlled by a state machine that interprets data from the camera and ToF sensors and enacts one to two possible states: patrolling and detecting

If the AI fails to detect a raven, the ToF sensors were calibrated to detect ravens based on proximity to the sensor(s), with a detection distance of 0.5 meters to allow for the robot to come to a complete stop before making contact with the raven

15 of 16

The robot uses the AI model and computer vision to detect ravens along a transmission line
Vision algorithms were trained on stock images and open source datasets
Data augmentation was performed to increase the diversity of the training data
~25,000 images were used for training the AI model
The model was tested on custom images and live video feed
The accuracy of the model was ~98%

Once a raven is detected, the system deters ravens using visual & audio stimuli
The robot drives along transmission line & relays information to the mobile app in real time
The robot is charged using Qi-wireless pad located at the bottom of battery compartment

PERFORMANCE SUMMARY

research deploy train evaluate

So, to summarize:��The robot is equipped with an AI model and computer vision which allows it to detect ravens along the transmission line. We trained the vision algorithms using stock images and open-source datasets, performing data augmentation to increase the diversity of the training data. We used around 25,000 images to train the AI model, and the model was tested on custom images and live video feed. The accuracy of the model was around 98%, making it an effective tool for detecting ravens accurately and efficiently.

Once a raven is detected, our system uses visual and audio stimuli to deter the ravens. The robot drives along the transmission line and relays information to the mobile app in real time, allowing the user to monitor the system's performance and effectiveness.

We have also implemented a Qi-wireless pad located at the bottom of the battery compartment for charging the robot. This feature allows for convenient and efficient charging, ensuring that the robot is always ready for deployment.��With its AI model, computer vision, and wireless charging capabilities, we believe that our robot will be an effective tool for preventing damage caused by ravens along power lines.

16 of 16

PLAN FOR NEXT REVIEW

Prototype to be tested in the field to evaluate effectiveness and accuracy of the vision system

More birds to be trained to avoid harm to protected species

Camera to be made 360 degrees by rotating to capture more data and minimize number of robots needed in a site

Material to be applied on the robot enclosure to ensure camera captures data in all weather conditions and make it weather-proof

Minor tweaks to be made to enhance weather resistance on the field

Mobile and web app to be calibrated based on the deployment location for testing and monitoring the system

research deploy train evaluate

As we move forward with the project, we have identified several key areas that we plan to focus on in order to continue improving the capabilities of the robot.

Firstly, we plan to test the prototype in the field to evaluate the effectiveness and accuracy of the vision system. This will allow us to make any necessary adjustments or improvements before deploying the robot in real-world situations.

Additionally, we plan to train more birds to avoid harm to protected species, and we will be rotating the camera to make it 360 degrees to capture more data and minimize the number of robots needed in a single site.

To ensure that the camera can capture data in all weather conditions, we plan to apply a material to the robot enclosure to make it weather-proof. We will also be making some minor tweaks to enhance weather resistance on the field.

Lastly, we plan to calibrate the mobile and web app based on the deployment location for testing and monitoring the system. This will allow us to ensure that the app is working effectively and efficiently in the specific environment where the robot will be deployed.

We look forward to sharing the results of our testing and improvements with you at the next review. Thanks a lot!