Research Paper

Affordable Gesture Controlled Robotic Hand and Forearm

By Chrysnell Destina

Background:

In this technologically advanced era, robotics has consistently demonstrated a significant impact on our society, particularly in the fields of engineering, medicine, and space science. Recognition of gestures has long been a big contributor to reducing the separation between real and digital worlds. Robotics demonstrated its crucial application in numerous industries such as military, defense, medical surgery, and industrial automation applications for pick-and-place functions. Gesture recognition is a strictly non-invasive approach to apply the user input for a wide range of actions (Salman et al. 2020). Gesture control technology has proven useful by saving time, increasing output, effectiveness, dependability, and avoiding personal injuries.

Many robotic arms perform tasks by mimicking gestures made by human hands. The applications for this arm include situations in which human hands alone are not efficient (Ss et al. 2016). Some of these conditions require extreme precision and seamless control. Gesture controlled robotic arms can be used to examine and disarm bombs, operate under radioactive environments, provide medical assistance and disinfection of hazardous materials, assist in space exploration, and surgical procedures as well as rehabilitation for patients with disabilities. They can also be used to control nuclear power plants and dispose of radioactive waste. Usages of robots can reduce errors and protect human lives in a variety of situations (Nair et al. 2018). Additionally, gesture-controlled robotic arms are often more affordable than other types of robotic arms, such as current industrial grade arms that could cost up to 400 thousand dollars. This version of a robotic arm is significantly cheaper, making it accessible to a wider range of people. This can be particularly important for organizations or individuals who may not have the financial means to purchase more expensive types of robotic arms. The accessibility of these robotic hands allows for greater use in a variety of settings, including in industries where precision and efficiency are key.

Hypothesis/Introduction:

In this experiment, I designed and 3D printed an advanced robotic hand and forearm using tough PLA filament. This hand has grasping capabilities and can be precisely controlled with a haptic glove that uses flex sensors to measure movement. This method of control is an improvement over traditional methods such as joysticks or remote controllers, as it allows for more intuitive and natural hand movements. The precision and utility of this hand and forearm make it an ideal choice for situations in which human safety is a concern, such as in firework manufacturing or bomb dispersal (Salman et al. 2020). The gesture-controlled hand is designed to closely resemble a human hand, which increases its usability and allows for more seamless integration into tasks that require delicate hand movements. Overall, this 3D-printed robotic hand and forearm represents a major advantage in the field of robotics and has the potential to revolutionize a wide range of industries.

Material and Methods:

Printing and assembling of the hand and arm

First, I 3D printed the base of the hand (fig 1) as well as its phalanges and forearm from an open sourced 3D printed robot project (Langevin et al. 2022) on a MakerBot Sketch 3D printer using PLA filament. After sanding all parts, they were attached to each other with a strong adhesive. The fingers of the hand are composed of 6 joint-like structures that combine to create a functioning finger (fig 2). Inside each joint is a hole for an elastic string to thread through to allow the finger to be pulled back simulating a contraction (fig 3). The hardware of the hand and forearm includes 5 servo motors with 180o degree rotation, each controlling a finger on the same hand. 1 string is attached to each servo motor and its corresponding finger. As the servo motors rotate, they pull the attached string, allowing for the contraction of the finger. The servo motors (fig 4) are also connected to an Arduino Nano (fig 5) that is placed in the hand, which receives signals from a haptic glove for control.

Fig. 1: 3D printed base of hand- photo by author Fig. 2: 3D hand & fingers- photo by author

Fig. 3: Finger with contraction strings- photo by author Fig. 4: Servo motors- photo by author

Fig. 5: Arduino Nano- photo by author Fig. 6: Haptic glove with flex sensors- photo by author

Programming of the hand

Arduino Nano RP2040 Connect is used to connect the 5 servo motors (fig. 4). To create the transmitter glove used to control the hand and forearm, a tactical glove was used as a base. This glove was made after programming and assembling the hand and forearm. 5 flex sensors attached to the glove are used to calculate the degree of movement of each finger. The flex sensor is a device that measures the motion or bending of an object. It is typically made of a combination of carbon and plastic. The resistance of the sensor changes when it is bent due to the connection between the carbon and the flex sensor (Jiang et al. 2027). These flex sensors send signals to an arduino nano board mounted to the glove. The flex sensors, attached to a glove (fig. 6) that the user will wear, controls the movement of the fingers at the end of the robotic arm. When the sensor is bent, the resistance value recorded changes and this leads to a different voltage value being transmitted to the Arduino (Jiang et al. 2017). This portion of the source code (fig. 7) has the function of translating input from the flex sensors into movement of the 3D printed fingers. Since the hand and forearm system is wirelessly operated, very few cables were needed and no main power supply was used. Instead, the individual modules were powered by two 9V batteries.

Fig. 7: Portion of source code (MertArduino et al. 2018)

Measuring Hand functionality

A crucial measurement to assess the gesture controlled robotic hand’s functionality is its precision and accuracy. One way to assess accuracy is by comparing the movements of the robotic hand with the movements recorded by the flex sensor glove. This can be achieved by analyzing the data from the flex sensor glove and comparing it with the intended movement of the robotic hand. The precision of the robotic hand can also be assessed by measuring the consistency and repeatability of its movements. This can be done by repeatedly performing the same task with the robotic hand and measuring the variability in its movements. Additionally, the sensitivity of the flex sensors can be measured to ensure that they are accurately detecting the user’s hand movements.

Another essential measurement is the response time of the robotic hand. It is important to record the time it takes the robotic hand to respond to the user's hand gestures. This ensures the robotic arm reacts in real time, making it efficient and effective. The range of motion is another critical factor that was measured. It is essential to determine the extent to which the robotic arm can move in various directions. This helps to establish the flexibility and adaptability of the robotic arm. The payload capacity is also a crucial measurement in the development of a gesture controlled robotic arm. It is essential to measure the amount of weight the hand can hold in different positions and tasks. This helps to establish the capacity of the robotic arm to perform various tasks. Finally, control sensitivity is a critical measurement that needs to be considered. It is necessary to calculate the minimal angle that each finger would respond to. This helps to establish the level of sensitivity and responsiveness of the robotic arm, making it more user friendly.

Data:

Measuring Hand functionality

Analysis and Conclusion:

The 3D printed gesture controlled robotic hand is a highly complex system that requires meticulous attention to detail to function correctly. One of the key issues that can cause the hand to fail is the coding involved. The coding is an integral part of the system and requires expert knowledge to get right. Any errors or faults in the coding can have a detrimental effect on the overall functionality of the hand. Thus, it is crucial to have a proficient programmer involved in the project who can ensure that the code is properly written and error-free. Another issue that can lead to the hand's malfunction is the calibration and connection of the flex sensors on the haptic glove. The flex sensors are responsible for detecting the user's hand gestures and translating them into commands that the robotic hand can understand. If the flex sensors are not correctly calibrated or connected to the hand, this can lead to unreliable data and inaccurate hand movements. Hence, it is important to ensure that the flex sensors are correctly calibrated and connected to the hand before testing.The brittle nature of 3D printed parts is another limitation that can affect the performance of the hand. The parts of the hand must be strong enough to withstand the forces generated during movement without breaking. If the 3D printed parts are too brittle, they may break easily, leading to frequent setbacks and delays in the testing process. Thus, choosing the right material for 3D printing and designing the parts with enough strength to withstand the forces generated is critical for the hand's success. Making connections on a breadboard can also be challenging, especially for those who lack experience in this field. It requires careful attention to detail, a steady hand, and good soldering skills. Any mistakes made during this process can lead to issues with the hand's electrical connections, which can affect its performance. Finally, issues with connecting the haptic glove to the hand via Bluetooth can also cause problems. Bluetooth connectivity is essential for transmitting the hand movements from the haptic glove to the robotic hand. If the connection is not established correctly, it can lead to inaccuracies in the hand's movements, which can affect its ability to perform its intended function. Ensuring a stable Bluetooth connection is therefore critical for the success of the hand.

The hand was unable to function correctly. However, the 3D-printed gesture-controlled robotic hand and forearm represent a major advancement in the field of robotics, with the potential to revolutionize a wide range of industries. The cost-effectiveness and accessibility of this version of a robotic arm make it accessible to a wider range of people and organizations, particularly those who may not have the financial means to purchase more expensive types of robotic arms.

Future Work:

There are several adaptations to my experiment that I plan to explore in the future, given more time and resources. One area of focus is the joint rotation of the hand's wrist, which would significantly increase the mobility and versatility of the hand. The ability to rotate the wrist would allow the hand to access a wider range of positions and orientations, making it more suitable for a variety of tasks and environments. Another potential modification is the addition of protective measures to the hand, such as insulation or shielding, to allow it to be used in hazardous or dangerous situations like bomb threats, handling harmful chemicals, and firework manufacturing. Another potential modification would be the implementation of Electromyography electrodes (where the electrodes would be on amputee) to control the hand wirelessly, as opposed to the haptic glove that is currently used (Zajdlik et al 2006). This would allow amputees with no hands to accurately control the 3D printed hand and forearm, making it more accessible for all people. Overall, these adaptations have the potential to greatly enhance the capabilities of my robotic hand and make it more useful in a wide range of applications

References:

Jiang, J., McCoy, A., Lee, E., & Tan, L. (2017) “Development of a motion controlled robotic arm.” IEEE 8th Annual Ubiquitous Computing, Electronics and Mobile Communication Conference (UEMCON), Vol. 2017, pp. 101-105.

Langevin, G. (October 25, 2022)“Open source 3D printed life-size robot.”(Version 1.0) [Source Code]. Retrieved on November 23, 2022 from https://inmoov.fr/inmoov-stl-parts-viewer/?bodyparts=Right-Hand

MertArduino (March 18, 2018)” Gesture control robotic hand source code” (Version 1.0), [Source Code]. Retrieved on December 5, 2022 from https://www.hackster.io/mertarduino/how-to-make-wireless-gesture-control-robotic-hand-cc7d07#cod.e

Nair, R., Kumar, S., Soumya, N., Shanmugasundaram, M. (March 2018) “A study on gesture controlled robotic arms and their various implementations.”International Journal of Mechanical Engineering and Technology (IJMET), Vol. 9(3), pp. 425–434

Salman, F., Cui, Y., Zafar, I., Liu, F., Wang, L., & Wu, W. (2020) “A Wireless-controlled 3D printed robotic hand motion system with flex force sensors.” Sensors and Actuators A: Physical, Vol.309, pp. 1-17.

Ss, Dheeban, Dhanasekaran Velayutha Rajan, H., A, Hari, Marimuthu, Prasanna, N, Senthil Kumar. (March 16, 2016) “Gesture controlled robotic arm.” The International Journal of Science & Technoledge, Vol. 4(3), pp. 101-112.

Zajdlik, J. (2006) "The preliminary design and motion control of a five-fingered prosthetic hand." 2006 International Conference on Intelligent Engineering Systems, London, UK, 2006, pp. 202-206