Wireless Data Glove Using Gesture Recognition Computer Science Essay

Published Date: 02 Nov 2017

ABSTRACT

Human-robot voice interface has a key role in many application fields. Robotics has achieved its greatest success to date in the world of industrial manufacturing. Hand gesture is a very natural form of human interaction and can be used effectively in human computer interaction (HCI). The approaches for analyzing and classifying hand gestures for HCI include glove-based techniques and vision based techniques. This paper discusses hand glove-based techniques that use sensors to measure the positions of the fingers and the position of the hand in real-time.

To implement this approach on real time application, the personal computer interface will be designed to control the movement of four degree of freedom (DOF) of didactic robot arm by transmitting the commands using wireless circuits. This paper focuses on wireless data gloves that are proposed to be used for gesture recognition and accordingly robot movement will take place.

Keywords:- Human-machine interaction, Wireless Data Glove, Accelerometer, PIC18F2510, RF 2.4 GHz transreceiver.

INTRODUCTION

The interaction between human and robot is definitely one of the major issues in the 21st century. This is due to the fact that, although, nowadays many tasks are being performed merely by robots, however, there are many cases [5, 6] in which robots either need the supervision and direction of a human being or they require collaboration with people to receive and process corresponding data to start a transaction or finish an assignment.

Robots are gaining an ever increasing foothold in society, with the particular uses of teleoperated robotic arms in fields such as medical surgery, remote manipulation of objects in hazardous environments, weaponry for warfare and industrial automation attracting a lot of research and development attention. It is envisaged that service and personal care robots will become more prevalent at home in the near future [8] and will be very useful in assistive operations for human care, particularly the elderly and disabled. Such robots are likely to become increasingly anthropomorphic in form and although many years will pass before such robots will become fully autonomous, there is still much to gain using robots controlled by humans.

Many challenges present themselves when it comes to the remote control of robots by humans, such as the ease of operation, haptic sensing, and telepresence. The problem of intuitive operation is mostly attributed to the type of interface available to the user, and the [6, 8] time taken to master the input controls. Haptic feedback requires that forces that robotic arm experiences are translated back to the operator in some sort of physical means, which greatly increases the ability to perform dexterous operations. Telepresence – where the operator is given the impression of being in the remote environment – is currently most often realized through the use of vision systems.

Hand gesture is a very usual type of human interaction and can be used efficiently in HCI. Feasibility of controlling home appliances using hand gestures presents an opportunity for a section of the aging population and disabled people to lead a more independent life [8]. The glove-based techniques use bend sensors to detect movement of fingers as well as magnetic/inertia tracking devices to track the pitch, yaw, roll and acceleration of the whole glove. The more sophisticated system which has multiple sensors such as bend sensors, abduction sensors and palm-arc sensors , wii remote sensor, Bio metric sensor, IR sensor optical sensor can be used to obtain accurate three-dimensional representations of the hand. The KHU-l data glove [3, 8] is capable of transmitting hand motion signals to a PC through wireless communication.

LITERATURE REVIEW

For hand-gesture recognition, some researchers have tried to perform the early segmentation process using skin-color histograms Zhou et al. [2] used overlapping sub-windows to extract invariants for gesture recognition, and characterized them with a local orientation histogram feature description indicating the distance from the canonical orientation. This makes the process relatively robust to noise, however, much more time consuming indeed. Kuno and Shirai defined seven invariants to do hand gesture recognition, including the position of the fingertip. This is not practically feasible when we have only pointing gestures, but also several other gestures, like grasping. However, the invariants they considered inspired us for our defined invariants. In some similar approaches, the watermark of an image is generated by modifying the invariant-vector. For example, Lizhong Gu and Jianbo Su tried to use Zernike moments along [2] with a hierarchical classifier to classify hand-gestures. This method is not appropriate for the JAST project, since there is not a high degree of freedom for the hands due to the limited space for movements and actions.

RESEARCH METHODOLOGY EMPLOYED

BACKGROUND

The design of a human body is so complex that to build a comparable machine, our technology will need to advance significantly whereby, with the current understanding and knowledge, engineers can only attempt to mimic the form of the human body. Thus, it is called the humanoid robot. A humanoid robot is an autonomous robot which has its overall appearance based on a human body. It can adapt to changes in its environment or itself and continue to perform what it is asked to. This is the main difference between humanoid robots and any other kind of machine robots which are usually used in the industrial fields.

This work addresses designing of a tele-operated anthropomorphic robotic arm and hand which consists of the robotic shoulder, elbow, arm, wrist and fingers. The robotic arm is designed to be similar to human arm with seven degrees of freedom where each part of the arm is actuated with servo motors. Besides, this design will be a versatile robotic arm system to give the robot the ability to manipulate objects for instance, simple pick and place operations. Moreover, many other components would need to be included to create a machine that would even come close to having the capabilities of a human arm. A data glove-based technique is used to to analyze and classify hand gestures.

DATA GLOVES

Sensor gloves are normally gloves made out of cloth with sensors fitted on it. Using data glove is a better idea over camera as the user has flexibility of moving around freely within a radius limited by the range of wireless connectivity. The glove is connected to a computer, unlike the camera where the user has to stay in position before the camera [5]. The effect of light, electric or magnetic fields or any other disturbance does not affect the performance of the glove.

Figure 1 Wireless Data Glove

The motion of a data glove is one of the most commonly used technologies for gesture-based human-computer interfaces. The motion data glove helps users to interface with the virtual world, and serves as an input device in non-verbal human computer interaction. Motion data glove is manufactured by attaching optical indicators onto an inexpensive consumer glove and using an optical motion capturing system for motion analysis.

HAND GESTURE

In some fields the interaction with human is inevitable. In entertainment, for instance, a good understanding of what people want is important. Imagine a robot during bomb detection where the supervision of an expert is needed to reduce the risk. Every specialist needs to know how to program a robot and insert the right instructions [2]. Thus, a natural way of interaction should be constructed so that the robot can obtain the relevant data from the surrounding people. Human-Robot Interaction or HRI addresses this need. Regarding a normal relationship between two (or more) people, they talk to each other, use gestures by means of parts and poses of their body, use the tone of their voice for stressing an issue or even make body-contacts like hand-shaking or patting on each other's back. Gestures are, in fact, used for everything from pointing at a person to conveying specific information or implying a message. It happens very often that one cannot simply express his or her feelings or opinions without using additional gestures.

APPROACH

The approach followed here is to enlarge the coverage and wireless connectivity to control robot arm by concentrating [1] over the R.F. circuitry and pulse width modulation scheme. This work addresses designing of a tele-operated anthropomorphic robotic arm and hand which consists of the robotic

Figure 2. Wireless Data Glove Microcontroller Program flowchart

shoulder, elbow, arm, wrist and fingers. The robotic arm is [6, 7] designed to be similar with a human arm with four degrees of freedom where each part of the arm is actuated with servo motors. Besides, this design will be a versatile robotic arm system to give the robot the ability to manipulate objects for instance, simple pick and place operations. Moreover, many other components would need to be included to create a machine that would even come close to having the capabilities of a human arm. The main approaches for analyzing and classifying hand gestures for HCI include glove-based techniques. Development of both hardware and software for a wireless hand gesture recognition system is the hand gesture recognition module design and the relevant software for the system. The main objective of designing a microcontroller is a large amount of electronics needed for certain applications can be eliminated.

Figure 3. Flowchart of Remote ARM Controller

1. ACCELEROMETER

An accelerometer is an electromechanical device which measures acceleration. The acceleration may be either static, like the constant force of gravity pulling at our feet or dynamic, either caused by moving or vibrating the accelerometer. Accelerometer would be used for motion and direction detection. The accelerometer is the device which will generate corresponding electrical signals to control the on-screen elements. The accelerometer used here is the 3-Axis accelerometer with an easy analog interface and running at a supply voltage of 3.3V, which makes it ideal for handheld battery powered electronics. The accelerometer will experience acceleration in the range of +1g to -1g as the device is tilted from -90 degrees to +90 degrees. In order to determine the angle of tilt, θ, the A/D values from the accelerometer are sampled by the ADC channel on the microcontroller. The acceleration is compared to the zero-g offset to determine if it is a positive or negative acceleration. This value is then passed to the tilt algorithm. When applied to all three axes, we are able to calculate the orientation of hand in three dimensional spaces.

Here the problem is restricted to only one degree of forehand movement. To accomplish the task a 3 axis analog accelerometer of free scale is used. (Only X-axis was utilized) The accelerometer is used as a tilt sensor. It is mounted on forearm such that whenever the forearm moves from one extreme position to the other, the accelerometer tilts on its X axis keeping Y and Z axis as a center. In the reference condition, when the accelerometer is aligned perpendicular to the X axis which was 0 degree tilt, it gives a constant DC reference voltage of 0.65 Volt for 5V power supply and zero g sensitivity. [7] As the accelerometer moves in +x direction the accelerometer output voltage starts increasing from 0.65V and as it completes 90 degree tilting the corresponding maximum voltage is 1.12V and on other angular motion for –X axis tilting, when it reaches to -90 degree the corresponding minimum voltage is 0.15V.As specified in the datasheet an RC filter with R=1.0 kΩ and C=0.1 μF on the outputs of the accelerometer is used to minimize noise Gravity.

Figure 4 Calculation Of Acceleration And Angle W.R.T

2. MICROCONTROLLER

The function of the microcontroller in this application is to act as an interpreter between the hand gestures and the end application. The ADC Port converts analog signals coming in from the accelerometer into corresponding 8-bit digital values. It then shifts out the result through the UART line to the device to be controlled. The PIC microcontroller having all these features achieves throughputs approaching 1 MIPS per MHz by executing powerful instructions in a single clock cycle, allowing the system designer to optimize power consumption versus processing speed. Microcontroller ports are used for interfacing with accelerometer, motor driver etc.

Position sensors are used to get the position of human hand interface. It is applied to microcontroller after signal conditioning. After processing information if send to remote robot arm through wireless link. Figure 3 and Figure 4 functional block diagram of remote robot arm. It includes RF receiver to receive the signals, it is connected with microcontroller vie serial link. Data from human interface board is processed and send it to driving circuitry.

Figure 5: Block Diagram Of Man And Machine Interface.

NON-INVERTING AMPLIFIER

A non-inverting amplifier is implemented to solve following two purposes:

A. IMPEDANCE/BUFFERING ISSUES

The PIC microcontroller datasheet specifies that for A-D conversion to work properly, the connected device must have output impedance under 10kΩ. The free scale analog accelerometer has an output impedance of 32kΩ. The solution is to use a low input offset rail to rail op amp as a buffer to lower the output impedance. LM324 was used in non-inverting configuration with gain of 3 to address the problem.

B. ADC REFERENCE VOLTAGE

For A-D conversion to work properly, the AREF of ADC must be more than 1.8V. But the accelerometer maximum DC output voltage was 1.12V. To have a proper A-D conversion the AREF must not be more then 1.12V, which counters the earlier argument. Hence the accelerometer output was amplified to 3.6V by a non-inverting amplifier to make AREF=3.5V.

Figure 9: Pulse Width Modulation

The PIC18F2510 each have 32 Kbytes of Flash memory and can store up to 16,384 single-word instructions. PWM has also been used in certain communication systems where its duty cycle has been used to convey information over a communications channel.

3. R.F. WIRELESS MODULE

➢ 2.4 GHz Carrier Frequencies

➢ 255 possible channels

➢ RS 232 UART interface with variable baud rate

➢ Standard configuration baud rate 9600

➢ Power LED indicator

➢ Input supply 5V to 12V

➢ 2 run modes: Packet mode and single byte transfer

➢ Variable packet length (0 to 40)

➢ programmable channel (0 to 255)

➢ Programmable device address: 255 per channel

➢ User friendly GUI for setting up RF module and

Test module

➢ compact Size, Plug and play

➢ On board EEPROM for saving settings

➢ Supported baud rate 300, 600, 1200, 4800, 9600

Figure 11: Wireless R.F Module

JUMPER SETTINGS

➢ CINFIG MODE

Closed: Configuration Mode

Open : Run Mode

➢ PACKET MODE

Closed: Variable Packet Length (with device address selection)*

Open: Single Byte Transfer (Broadcast) 80msec delay between 2 char)

CONFIGURATION MODE

Table No. 1. R.F Module Configuration Parameters

Once you enter in to configure mode you can set 3 values for your module. GUI looks like this where you can set all this values

Figure 12. GUI For R.F. Module Configuration

RUN MODE

When module is in run mode it can operate in 2 modes

4. EXPERIMENTS AND RESULTS

The experiment is based on wireless data glove which consists of accelerometer, PIC Microcontroller and R.F 2.4 GHz wireless module. The analog signals generated by accelerometer are proposed to be used in ADC of PIC microcontroller and then gets converted in to digital form. This digital signal is then given as input to tilt sensing algorithm. These are the programmes developed for PIC microcontroller. This algorithm generates the output which is sent [4] through RF module to Personal Computer Interface for accomplishing the final task. The output is shown in the table no.5

Figure 13 shows the two commands for manual arm control, auto arm control. After click on manual arm control the next screen is shown in Figure13.

Figure14. The Result Of Manual Arm

Control Command

Figure14 shown the result when manual arm control command will executed. Above figure shows the five types of buttons for left, right, up, down movement of robotic arm. Exit button is for exit from program when program is running or executing.

5. CONCLUSION & FUTURE SCOPE

This paper discusses the hand gesture recognition module design for analyzing and classifying hand gestures for HCI including glove-based techniques.

Hand Gesture Recognition using Wireless Data Gloves system can be used to solve the problem in supervisory control. It is used to find the way to map a set of angular measurements as delivered by the data glove to a set of predefined hand gestures. Furthermore, it would be advantageous to have a system with a certain amount of flexibility, so that the same system could be used by different people for performing various varying sets of tasks.