The Sixth Sence Technology

Published Date: 02 Nov 2017

ABSTRACT:

Sixth Sense is portable gestural interfaces, with the physical world around us with digital information increases and let our natural resources gestures to interact with that information.

Although miniaturization of computers allows us to carry out the world in our pockets we can enter into the digital world we have to adapt a bridge between our digital devices and our interactions with the physical world. Information is traditionally confined to paper or digitally on a screen. SixthSense bridges this gap, allowing intangible digital information tangible world, and allowing us to interact with this information via gestures. Sixth Sense frees information from its confines by seamlessly integrating with reality, and thus makes the world team.

The Sixth Sense prototype implements several applications that demonstrate the utility have the ability and flexibility of the system. The browser application allows the user navigate a map displayed on a nearby surface using hand gestures, similar to gestures supported systems based on Multi-Touch, which allows the user to zoom in, zoom out and move with intuitive hand movements. The drawing application flexibility help attract based on any surface by tracking the movements of the fingertip of the index finger of the user. The user can stop by any surface or wall and flick through the photos you have taken.

SixthSense also lets the user draw icons or symbols in the air with the movement of the index finger and recognizes those symbols as interaction instructions. If you draw a magnifying glass symbol takes the user to the map application or drawing an "@" symbol to check your mail. The SixthSense system also increases physical objects the user is interacting with by projecting more information about these objects projected on them. Newspaper reader can show live video news or dynamic information can be provided on a sheet of plain paper.

CONTENTS:

Abstract

Introduction………………………………………..…..6

Related technologies…………………………..………7

Augmented reality…………………………………..…7

3.1Head-mounted displays…………………………….7

3.2Display of hand…………………………………….7

3.3Spatial screen………………………………………8

Gesture recognition…………………………………...8

4.1Symbolic gestures…………………………………9

4.2Deictic gestures…………………………………...9

4.3Ionic gestures……………………………………..9

4.4Pantomimic gestures……………………………...9

Vision based algorithm………………………………9

Sixth sense technology as a main platform………....10

6.1Radio frequency identification…………………..10

Working of sixth sense technology………………....11

7.1Camera…………………………………………...11

7.2Projector…………………………………………12

7.3Mirror………………………………………….…12

7.4Mobile component……………………………….12

7.5Colour markers…………………………………...13

Sixth sense technology……………………………....13

8.1What is sixth sense……………………………….13

8.2How was sixths sense earlier prototype…………..14

8.3What sixth sense prototype is now………………..15

9.Working16

10.Applications…………………………………………...18

10.1Make a call………………………………………..18

10.2Call up a map……………………………………...18

10.3Check the time……………………………………..19

10.4Create multimedia reading experience…………….19

10.5Drawing applications………………………………19

10.6Zooming features…………………………………..20

10.7Get product information……………………………21

10.8Get book information……………………………….21

10.9Get flight updates…………………………………...21

10.10Feed information of people………………………..21

10.11Take pictures………………………………………22

10.12Check the email……………………………………22

11.Advantages………………………………………………22

12.Feature enhancements…………………………………...22

13.Conclusion……………………………………………….23

14.References………………………………………………..23

List Of Figures

Figure 7.1: Camera

Figure 7.2: Projector

Figure 7.3: Mirror

Figure 7.4: Mobile component

Figure 7.5: color markers

Figure 8.1: sixth sense

Figure 8.2: Earlier Device

Figure 8.3: present prototype

Figure 9: working principle of sixth sense technology

Figure 10.1: Making a call

Figure 10.2: Call up a map

Figure 10.3: Drawing the time

Figure 10.4: Experiencing multi media

Figure 10.5: Drawing a picture

Figure 10.6: Zooming features

Figure 12.7: information about book

Figure 10.8: About flight updates

Figure 10.9: Information about person

Figure 10.10: Making pictures

1.INTRODUCTION:

We have evolved over millions of years to sense the world around us. When we point out something, someone or some place we use our five natural senses, which include the ears, nose, tongue, eyes of the mind and body to perceive the information, that information helps us make decisions and chose the right actions to take. But undoubtedly the most useful information that can help you make the right decision is not naturally perceivable with our five senses, mostly the data, information and knowledge that mankind has accumulated about everything and which is increasingly available in online.

Although the miniaturization of computing devices allows us to carry computers in our pockets, making connection digital world every time, we have to adapt link between our digital devices and our interactions with the physical world. Information is setting up to paper or digitally on a screen. SixthSense bridges this gap, bringing the intangible digital information the tangible world, and making free to interact with this information via natural gestures. SixthSense adapt information from its confines by seamlessly integrating with reality, and thus make the world whole team.

"Sixth Sense Technology" is the new slang has announced its presence in the technical field. This technology has emerged, which has its connection with the power of the six directions. Our normal computers will soon be able to detect the different feelings accumulated around and everything is a gift of the "Sixth Sense Technology" recently introduced.

SixthSense is a "gesture based" portable device that increases the physical world with digital information and lets people use natural hand gestures to interact with that information. Pranav Mistry, a PhD student in the Fluid Interfaces Group at MIT Media Lab, developed it. A graduate student with fluid interfaces group at MIT, which caused a storm with his creation of SixthSense. He says that the film "Robocop" and "Minority Report" gave him the inspiration to create his vision of a world that is not dominated by computers and human robots, where computers and other digital devices enhance the enjoyment of the physical world of people.

2.RELATED TECHNOLOGIES:

SixthSense Technology has a different approach to the computer and tries to make the look of our digital lives more intuitive, interactive and above all, natural. We should not have to think about it separately. It's a lot of complex technology put in a single portable device. When we bring connectivity, we can get instant visual feedback, relevant projected onto any object that collect or interact with technology is mainly based on the augmented reality, gesture recognition, the algorithm based on artificial vision.

3.AUGMENTED REALITY:

Augmented reality is a term for a direct or indirect view of a live physical real-world environment whose elements are complemented by virtual computer generated imagery. It is related to general concept called mediated reality in which a view of the changed reality by a computer. The increase is conventionally in real time, and the semantic context with environmental elements.

Sixth sense technology that uses Augmented Reality concept to super impose digital information about the surrounding real world of the user becomes interactive and digitally helpful. Artificial information about the environment and the objects can be stored and retrieved as an information layer on top of the real world view.

The main hardware components for augmented reality are: display, monitor, input devices, and computer. The combination of powerful CPU, camera, accelerometers, GPS and a solid-state compass are often present in modern Smartphone, which makes possible platforms.

There are three major techniques for augmented reality

3.1Head Mounted Displays:

A Head Mounted Display (HMD) places images of both the physical world and registered virtual graphical objects on the views of users worldwide. The HMD optics are or video see through or see through nature.

3.2Display of hand:

Augment Reality laptop uses a small computer device with a screen that fits in a the user's hand. All handheld AR solutions to date have employed techniques see-through video to overlay graphical information with the physical world. Initially hand AR uses sensors such as digital compasses and GPS units for sensors six degree of freedom tracking.

3.3Spatial Screens:

The user must carry the display such as with head mounted or screens Portable devices spatial Augmented Reality (SAR) makes use of digital projectors to display graphical information onto physical objects.

Mobile augmented reality systems use one or more modern of the following tracking technologies digital cameras or other optical sensors, RFID (Radio Frequency Identification), wireless sensors, etc. Each of these technologies has different levels of accuracy and precision. Most important is the monitoring of posture and position of the user's head to increase the user's view.

4.GESTURE RECOGNITION:

Gesture recognition is a topic of computer science and language technology with the goal of interpreting human gestures via mathematical algorithms. Gestures can be any movement or able to recognize, but generally originates in the face or hand. Current field include emotion recognition from the face and hand gesture recognition. We can make with cameras and computer vision algorithms to interpret sign language. Many approaches

Gestures can exist in isolation or include external objects. Free of any object to wave invite, defend, and to a greater or lesser extent, make use of more formal sign languages. With respect to the objects, we have a wide range of gestures that are almost universal, including pointing to objects, touching or moving objects, object shape change, activation of objects such as controls, or deliver objects to others.

Motion detection can be considered as a way for computers to begin with the human body to understand the language and therefore the richest among the machines and primitive humans or text user interfaces (GUIs) GUIs always the keyboard and mouse input limit. Gesture recognition makes human-machine interface (HMI) and interact naturally without mechanical devices.

4.1Symbolic gestures:

These are gestures that, within each culture must become single meaning. An emblem as a gesture of "OK" is a case in point, the gestures of American Sign Language but also fall into this category.

4.2Deictic gestures:

These are the types of gestures usually are observed in HCI and pointing gestures or otherwise direct the listener's attention to specific events or objects in the environment.

4.3Iconic gestures

As the name suggests, these gestures are used to convey information about orientation of the object that is discourse. Are the gestures made when someone says? The plane flew like this, while moving his hands in the air like the flight path of the aircraft.

4.4Pantomimic gestures:

These gestures are typically used in showing the use of some invisible movement

Tool or an object in the hand of the speaker. When an announcer says: "I took a sharp turn to the left" while mimicking the action of spinning a wheel with both hands, they are doing a gesture pantomime.

Using the concept of gesture recognition, it is possible to point the finger at the computer screen so that the cursor moves. This can make conventional input devices such as mice, keyboards and touch screens even redundant. Motion detection can be performed using machine vision and image processing techniques. The bibliography includes ongoing work in the field of computer vision in capturing gestures or more general human postures and movements of the camera connected to a computer.

5.VISION BASED ALGORITHM:

Computer vision is the science and technology of machines. As a scientific knowledge computer vision is concerned with the theory of artificial systems that extract information from images. The image data can take many ways such as views from multiple cameras, video sequences, and multi-dimensional data from a medical scanner.

Computer vision, on the other hand, describes the processes implemented in software and hardware behind artificial vision systems. The tracking software user gestures using algorithms based on computer vision. Computer vision is, in a sense, the inverse of the computer vision can produce 3D models from image data and computer graphics can produce image data from the 3D models of the two fields, for example, as discussed in augmented reality.

The fields most closely related to computer vision are machine vision, image analysis and image processing. Both Image processing and analysis tend to focus on 2D images, how to transform one image to another. Characterization involves processing image analysis not require assumptions or interpretations producing on image content. Machine vision tends to focus on the 3D scene is projected onto one or more images, for example, how to rebuild the structure or other 3D scene information from one or more images. The vision tends to focus on applications, mainly in manufacturing, vision based autonomous robots and systems for inspection or vision-based measurement.

6.SIXTH SENSE TECHNOLOGY AS MAIN PLATFORM:

SixthSense Technology has a different approach to the computer and tries to make the look of our digital lives more intuitive, interactive and above all, natural. When you bring in connectivity, you can get instant visual feedback, relevant projected on any object of pick up or interact with. So, pick up a box of cereal and screened device if it suits your preferences. Some of the technologies that use this are radio frequency identification, gesture games, washing machine.

7.RADIO FREQUENCY IDENTIFICATION:

SixthSense is a platform for RFID (Radio Frequency Identification) based business intelligence that combines RFID events with information from other enterprise systems and sensors to automatically make inferences about people, objects, workspaces and their interactions.

Radio frequency identification technology is basically an electronic labeling which enables the detection and tracking labels and therefore objects that are attached to. This ability to do remote sensing and monitoring along with the low cost of passive tags has led to the widespread adoption of RFID in supply chains worldwide.

Pranav Mistry, a researcher at the Media Lab of the Massachusetts Institute of Technology, has developed a device of "sixth sense" - a device worn on the wrist that can function as a "touch screen" for many modern applications. The gadget is able to select a product, either by image recognition or radio frequency identification tags (RFID) and project information, as well as an Amazon rating.

SixthSense The idea is to use radio frequency identification technology in conjunction with a group of other enterprise systems such as calendar system or online presence that can track user activity. In this sense, we consider the future business environment in which people (or rather their employee badges) and their personal objects such as books, laptops and mobile phones, tagged with cheap passive RFID tags, and there is good RFID readers coverage in the workplace.

SixthSense incorporates algorithms that start with a mass of differentiated labels and automatically infer a range of data based on an accumulation of observations. The technology can automatically distinguish between people and tags labels of objects, learn the identities of people infer the ownership of objects by people, learn from the nature of the different areas of a workspace, and perform other such inferences.

By combining the information from these different sources, SixthSense records all tag level events on a raw database. Inference algorithms consume these events without inferring events at the level of people, objects and workspace areas, which are recorded in a database processed separately. Applications can probe these databases using SQL query execution or creating triggers to be notified of events of interest.

SixthSense inferred when a user has interacted with an object, for example, when you pick up your phone. It is a platform on which its programming model makes inferences automatically available to applications through a rich set of APIs. To demonstrate the capabilities, researchers have prototype some applications using these APIs, including a service misplaced object alert, enhanced calendar service, and rich annotation of video with physical events.

7.WORKING OF SIXTH SENSE TECHNOLOGY:

7.1Camera

A camera captures and recognizes an object to the eye and hand gestures follow user-using techniques based on vision on computer.

Figure 7.1: Camera

Send data on your smartphone. The camera, in a sense, acts as a digital eye, seeing what you see. It also keeps track of the movements of the thumbs and fingers of both hands of the user. The camera recognizes objects around you instantly, with the micro-projector overlaying the information on any surface, including the object itself or your hand.

7.2Projector

Send data on your smartphone. The camera, in a sense, acts as a digital eye, seeing what you see. It also keeps track of the movements of the thumbs and fingers of both hands of the user. The camera recognizes objects around you instantly using micro projector on any surface, including your hand we can projects the information.

Figure 7.2: Projector

7.3Mirror

The use of the mirror is significant because the projector pointing downward hangs from the neck.

Figure 7.3: Mirror

7.4Mobile Component

Mobile devices like Pocket Smartphone in transmission and reception of voice and data anywhere and to any person through mobile Internet. A deputy Smartphone running the SixthSense software, and is responsible for connecting to the Internet. A smartphone web-enabled user's pocket processes the video data. Other web searches can interprets hand gestures.

Figure 7.4: Mobile component

7.5Color Markers

Is at the fingertips of the user. Marking the user's fingers with red, yellow, green and blue belt helps the camera to recognize gestures. The arrangements of these manufacturers are interpreted into gestures that act as interaction instructions for the projected application interfaces.

Figure 7.5: color markers

8.SIXTH SENSE TECHNOLOGY:

8.1What is Sixth Sense?

Figure 8.1: sixth sense

Sixth sense in scientific terms is defined as the extra sensory perception or ESP order. Reception is no information obtained through any of the five senses. Nor is taken from some past experience or known. Sixth Sense aims to more seamlessly integrate online information and technology in everyday life. By providing information necessary for decision making beyond what we have access to with our five senses, effectively gives users a sixth sense.

8.2How was sixth sense earlier prototype?

Figure 8.2: Earlier Device

Maes MIT group, which includes seven graduate students were thinking about how a person could be more integrated into the world around them and access to information technology sixth sense without having to do something like make a phone. At first there was a bracelet to read a RFID tag to know, for example, the books that a user is holding a store.

They also had a band that uses infrared to communicate by beacon to supermarket smart shelves to give you information about products. As we grab a packet of macaroni, the ring will light up red or green to tell us whether the product was free of trace organic or peanut whatever criteria the program in the system.

They wanted to make the most useful information for people in real time with minimal effort in a way that requires no behavioral change. The bracelet was coming, but still had to get our cell phone to view the information.

Was then hit on the idea of ​​accessing the Internet and project information. So someone wearing the bracelet could pick up a paperback book in the library and call your comment about the book, projecting on a surface in the store or do a search for keywords through the book to access the scanned pages Amazon or Google books.

Started with a larger projector that is kept on a helmet. But that proved large and heavy if someone was projecting data into a wall and then turned to speak to a friend the data is projected onto the face of his friend.

8.3WHAT SIXTH SENSE PROTO TYPE IS NOW:

Figure 8.3: present prototype

Now they have changed to a smaller projector and created the prototype roof that is worn around the neck.

The Sixth Sense prototype is comprised of a portable projector, a mirror and a camera. The hardware components are coupled in a portable device such as mobile pendant. Both the projector and the camera are connected to the mobile computing device in the user's pocket.

We may well consider the Sixth Sense Technology as a mixture of computer and cell phone. It functions as the device attached to it hangs around the neck of a person and therefore the projection is initiated by means of micro projector connected to the device. Therefore, in the course become moving equipment itself and fingers act as a mouse and keyboard.

The prototype was built from a common webcam and battery powered 3M projector, with a mirror attached all connected to an Internet-enabled mobile phone. The facility, which costs less than $ 350, allows the user to project the phone information on any wall surface, the body of another person or even your hand.

Mistry wore the device on a string around his neck, and pen caps on four fingers colors dark colors helped the camera distinguish the four fingers and recognize his hand gestures with software that Mistry created.

9.WORKING OF SIXTH SENSE TECHNOLOGY:

Figure 9: working principle of sixth sense technology

The entire hardware device is included on a mobile handheld device shaped pendant. Basically, the camera recognizes individuals, images, pictures, gestures you do with your hands and assists projector to project any information on any type of surface is present in the person's forehead. The use of the mirror is significant because the projector pointing downward hangs from the neck. To carry out variations on a much higher plane in the demo video, which was broadcast to show the prototype to the world, Mistry uses colored caps with your fingers to make it easier for the software to differentiate between the toes requiring multiple applications.

The software analyzes the video data captured by the camera and tracks the locations of the colored markers using computer vision techniques individual. One can have any number of gestures and movements as long as all of them are identified and differentiated for the system to interpret reasonable, preferably through fiducially unique and varied. This is only possible because the device of 'Sixth Sense' supports multi touch, multiuser interaction.

MIT basically plans to increase reality with a pendant picoprojector hold an object in the store and explosions device relevant information about it (such as environmental statistics, for example) can be searched and manipulated with hand gestures. The "sixth sense" in question is the Internet, which provides a natural data, and that can be almost anything - MIT has shown off the device projects information on a person at a party meet in real person (in pictured), projecting flight status in a boarding pass, along with a non-contextual interface to read the entire email or make calls. It is a very interesting technology that, like many of the projects of the MIT Media Lab, makes the wearer like a complete idiot - if the projector does not give it away; the bands of color finger using the device to detect motion of the fingers could certainly.

The idea is that the sixth sense is to determine how the people are using the device and how they are using this device. The software can use the Internet for information that may be relevant to the situation, and then the projector takes over.

Dr. Maes says all is with in the software. The system is constantly trying to find what's around you, and what we're trying to do. You have to recognize the images you see, track your gestures, and then relate everything to relevant information at a time ".

Mainly how it works it creates as follows:

The hardware that makes the sixth sense is the work of a pendant like mobile portable interface.

There is camera, a mirror and a projector and connects wirelessly to a Bluetooth smartphone or 3G or Wi-Fi that can slide comfortably in ones pocket.

The camera captures individuals, images, pictures, gestures are made with hands.

The information is sent to the smartphone for processing.

Projects that looks down projector output image in the mirror.

Mirror reflects the image onto the desired surface.

Therefore, the digital information is released from its confines and placed in the physical world.

10.APPLICATIONS:

This prototype has several applications that demonstrate the usefulness, viability and flexibility of the system.

Sixth Sense device has boundary less applications some of the important applications are as fallows

10.1Make a call

You can use the sixth sense to project a keyboard in his hand, and then use that virtual keypad to make a call. Calling a number also will not be a big task with the introduction of Sixth Sense technology. Not require mobile, just type the number in the palm acts as a virtual keyboard. The keys will come up on your toes. The fingers of the other hand will be used to enter the number and call.

Figure 10.1: Making a call

10.2Call up a map

The map also implements sixth sense that allows the user to view the map in any physical surface and find your destination and you can use your thumbs and forefingers to navigate the map, for example, to zoom in and out and making other controls.

Figure 10.2: Call up a map

10.3Check the time

Sixth Sense all we have to do is draw a circle on your wrist with the index finger for a virtual clock gives the correct time. The computer is the top red marker, which recognizes the gesture and sends data to projector and projector will flash the image of a clock your wrist.

Figure 10.3: Drawing the time

10.4Create multimedia reading experience

The SixthSense system also increases physical objects the user is interacting with by projecting more information about these objects projected on them. A newspaper reader can see live video news or dynamic information can be provided on a regular basis.

Figure 10.4: Experiencing multi media

10.5Drawing application

This application user can draw their own on plain surface by tracking the movements of the fingertip of the index finger of the user.

Figure 10.5: Drawing a picture

10.6Zooming features

The user can zoom in or zoom out with intuitive hand movements.

Figure 10.6: Zooming features

10.7 Get product information

Maes says Sixth Sense uses image recognition or marker technology to recognize products you pick up, then you feed information on these products. For example, if you are trying to shop "green" and are looking for paper towels with the least amount of chlorine in them, the system will search for the product you pick up from the shelf and gives guidance as to whether this Product is a good choice for you.

10.8Get book information

Maes says Sixth Sense uses image recognition or marker technology to recognize products you pick up, and then you feed information about books. The system can project Amazon book ratings and reviews and other relevant information

Figure 10.7: information about book

10.9Get flight updates

The camera capture and search in the web and let you know if your flight is on time or any delay.

Figure 10.8: About flight updates

10.10Feed information on people

When you go out and meet someone, project information, such as what they do, where they work as well as m could represent variables on the person floating on his shirt. It might be useful if you view the status of the relationship of Facebook to let you know not to waste your time

Figure 10.9: Information about person

10.11Take pictures

If fashion the index finger and thumb in a square the system takes a picture. After you have the desired number of photos, we can project them on a plain surface, and use gestures to nice way of photos, and arrange and resize.

Figure 10.10: Making pictures

10.12Check the email

We check email by developing '@' symbol your post clearly shows where you can be connected and you can find the information.

11.ADVANTAGES:

Normally we have five senses, after progression of sixth sense easy to use interface that integrates digital information in the physical world and its objects, so that the world whole team.

This technology will not change any human habits, but the causes of computer and other machines to suit human needs.

Use gestures to interact with digital information.

Compatible with multi user interaction and multi-touch.

Any person without any basic knowledge of a keyboard or mouse could use the device.

No need to carry a camera more. if you go on vacation, then from now on wards will be easy to capture photos using only your fingers.

This device can be used by anyone without even a basic knowledge of a keyboard or mouse.

It is portable and easy to carry, and that we can use in our neck.

It is gesture controlled computing device that feeds our relevant information and turns any surface in relation to the screen.

12.FUTURE ENHANCEMENTS:

Have a tracking system with 3D gesture

We can use this new technology in various interests, such as gambling, education systems.

To secure mobile devices, such as we use for the sixth sense, as projector and camera.

To get rid of the colour markers.

13.CONCLUSION:

It is a new global dimension for anyone that sixth sense objects recognizes objects around you, displaying information automatically and letting you access it in any way you want, the easiest way possible.

Everyone in this world wants to be new, it has to be very fast and has good future to progress whatever is uni dimensional in society of users are becoming transparent user interface that is tangible to access information if you can eliminate colour markers and fingertips and caps. Needless to develop the initial stages, which has a vast exposure of this and clear knowledge of it and must be fully aware about the latest technology, it is good to use for each.