Getting Started With Intel Realsense 3D Technology

Introduction

Imagine a scenario where a user can interact with your app just like they interact with people, you provide your app human-like senses and the user can interact with them with natural gestures and voice commands. This is what Intel Realsense Technology is all about. It provides developers with many new opportunities.

Probably you have gone through my previous article.

• Intel Realsense 3D Technology – An Overview

And are excited to get started with Realsense 3D camera. This article will help you with that.

Intel RealSense Technology

Figure 1: Intel RealSense Technology

Intel Realsense Technology provides human-like senses to devices capable of hand and finger tracking, speech recognition and synthesis, facial analysis, background segmentation, fully textured 3D scanning and enhanced depth augmented reality and much more. It is a combination of Intel Realsense 3D camera and software. It was previously known as perceptual computing.

Hardware and Software Requirements

1. Intel RealSense 3D Camera (Front F200)
2. Download F200 Intel RealSense Depth Camera Manager
3. Download the Intel RealSense Software Development Kit
4. JavaScript Enabling Package for Intel RealSense SDK version R3(Optional)
5. Redistributable components and additional speech language packages (Optional)
6. 4th generation Intel Core Processor with minimum 8 GB free hard disk space.
7. Windows 8.1 64 bit Operating System

Setting up the Environment

Connect your Realsense 3D Camera to a USB 3.0 port of your computer. The OS does not detect the camera if it is connected to a USB 2.0 port. The supported operating system is Windows 8.1, but it works fine on Windows 10. There are three flavors of Realsense cameras available.

1. Intel RealSense 3D Camera (Front F200)

Figure 2: Intel RealSense 3D Camera Front F200

The camera faces the user and is designed to sense a user’s head and hands in 3 dimensions and allows users to interact more naturally with their computers using gestures and even facial and voice recognition.

Features

• Full hand-skeletal tracking and gesture control
• 3D segmentation
• Facial analysis
• Depth-enhanced Augmented Reality
• Speech
• 3D Capture for faces

2. Intel RealSense Snapshot

Figure 3: Intel RealSense Snapshot

The camera gives the user the ability to do more with, and get more out of, their pictures. You can change the focus after taking a picture or can measure the distance directly by drawing lines on your device. It brings photography to the next level for everyone.

Features

• Focus and color
• Motion photo filters
• Measurements capture

3. Intel RealSense 3D Camera (Rear R200)

Figure 4: Intel RealSense 3D Camera

The camera typically faces away from the user. It was designed to sense the environment around it in 3 dimensions.

Features

• 3D capture: faces, people and environments
• Depth-enhanced Augmented Reality
• Depth-enhanced photography and video
• Measurement
• Face detection and tracking

I have an Intel RealSense 3D Camera (Front F200). You can get yours from here for $99 and it ships to most parts of the world or you can buy a system with an integrated Realsense F200 Camera. You need to have both the Realsense camera and the SDK to enable apps with Realsense Technologies.

Figure 5: Realsense development Kit

1. Setup the Intel Realsense Developer Kit Camera: Position and install your camera to the top of the computer or laptop and connect the USB of the Camera to USB 3.0 port of your system. If the camera is integrated into the computer or laptop you can directly install the SDK and Camera Manager.
2. Install Intel RealSense Depth Camera Manager.* If your Realsense camera is not connected then the setup will be interrupted.Figure 6: Intel RealSense Depth Camera Manager
    
3. Run the Intel RealSense Software Development Kit installer. When downloading you need to fill in your email address and other details even if you have registered with Intel before. For installation you will see a welcome screen, follow the instructions to complete the installation process. If you have any previous version of SDK installed, the installer will prompt you for an upgrade but it is always recommended for a clean uninstall and then fresh install.*By default, the SDK installs to the C:/ProgramFiles(x86)/Intel/RSSDK directory.*After installation, reboot your system to propagate all the environmental variables.
   Figure 7: SDK Installation
    
4. You can also install the language packs if you are using languages other than English.
5. Be in a relaxed position in front of your camera so that your hands can move freely. Now you are ready to play around with Intel Realsense 3D Technology. Let us try some sample apps.

Intel Realsense SDK Sample Browser

Intel Realsense SDK Sample Browser provides many samples with complete documentation and source code. You can play around with these sample in C++, C#, Unity, Processing, JavaScript and Java languages. For C# the .Net framework 4.0 is required. The IDE used in the samples is Visual Studio 2010 or later and Unity 4.0 or newer.

Figure 8: Intel Realsense on Start Menu

Go to Desktop, Intel RealSense SDK and then Intel Realsense SDK Sample Browser or you can search directly in the Metro Tiles from your Start Screen. Open it.

Figure 9: Intel Realsense SDK Sample Browser

Let us try the Emotion Viewer C# sample.

The Emotion Viewer Sample is a Windows Desktop application written in C#. It will detect your emotions by tracking your face and display them next to your face.

Figure 10: Displays emotions next to your face

Face Tracking Demo (Unity)

Face Tracking Demo shows a 3D avatar that mimics your facial expressions. To run this sample, Unity 4.3.4 Pro or newer is required. You can toggle between some 3D avatar by pressing I.

Figure 11: Face tracking in unity

Hands Viewer (Unity)

This sample tracks every movement of your hands and also shows that in corners. There is a 3D model that mimics your hand movements. To run this sample, Unity 4.1.0 Pro or newer is required.

Figure 12: Hands Viewer in Unity

*Realsense Camera installed on my 11.6” Ultrabook

Voice Synthesis (C++)

The voice synthesis sample shows text to speech translation. You can play around with Pitch and Speech Rate and also can use language other than English by downloading language packs.

Figure 13: Voice Synthesis

Points of Interest

There are many more samples to play around with. Also, you can check the source code from which you can learn very much and then you can start experimenting with Realsense Technology.

History

v1 of article.

References

• Developing for the Intel® RealSense™ 3D Camera (F200)
• Intel® RealSense™ SDK
• Intel® RealSense™ SDK 2014