【PeterShen_Skysim】Leap Motion 探究【第一篇】-小明编程

本篇文章由PeterShen_Skysim所作，我转载到这里，如有疑问请点击原文和他交流~

文章对应的代码由他特别提供，点击这里跳转到csdn下载。

这是一篇C#开发LeapMotion SDK的代码，之前遇到好几个人说找不到C#教程，因此特地转载到这里。

由于团队开发需要，今天拿到了Leap Motion做测试开发，也就是历动，一款手部识别传感器。
拿到历动之前已经对它有所了解，然而拿到手后发现确实不如想象中的那么没好，由于基础图像识别，肯定有一些弊端，例如手部遮盖部分识别出错，应用体验一般，应用也比较少等，给我的感觉好像这是一款还处于最后优化的产品，并不能代替现有的交互操作体验。不过，在一些简单的交互上，Leap还是给我了一个很好的反馈，比如手掌的左右倾斜，手指简单的点击操作等。
结合VR交互，这款产品应该是一个颠覆性体验，抛弃了传统的遥控器式手柄，完全手部操纵，所见即所得的感觉，冲击还是很大。所以打算开一个专题，记录下开发Leap的点滴。

万事当先，Leap提供了很友好的SDK！这对开发者极其重要，不知道牛长什么样怎么去喂牛？Leap支持的平台有：C, C#, Unity, Object-C, Java, Python, JavaScript, Unreal Engine。非常庞大的支持了，然而百度搜索到的开发文档还是寥寥无几，谷歌到的东西也不多，不知道是体验不够优秀，还是什么原因，这么好的一款产品应该大家一起来优化才对。

由于最近.net平台开发比较多，所以还是以CS做例子，看一下官方的源码，然后写一个自己的例子看看怎么处理消息。

主程序段很简单，首先实例化两个类，SampleListener（这个类后面详细讲，是Leap的核心部分）和Controller，然后给Controller增加一个监听，监听SampleListener反馈的消息，和EventHandle其实是一样的。关闭时注意需要停止监听，并且丢弃Controller。

下面开始讲SampleListener，即Leap中的Listener，不过由于Listener内消息都要自己编写，所以必须创建一个新的Listener去override Leap中的Listener。

class SampleListener : Listener
{
	private Object thisLock = new Object ();

	private void SafeWriteLine (String line)
	{
		lock (thisLock) {
			Console.WriteLine (line);
		}
	}

	public override void OnInit (Controller controller)
	{
		SafeWriteLine ("Initialized");
	}

	public override void OnConnect (Controller controller)
	{
		SafeWriteLine ("Connected");
		controller.EnableGesture (Gesture.GestureType.TYPE_CIRCLE);
		controller.EnableGesture (Gesture.GestureType.TYPE_KEY_TAP);
		controller.EnableGesture (Gesture.GestureType.TYPE_SCREEN_TAP);
		controller.EnableGesture (Gesture.GestureType.TYPE_SWIPE);
	}

	public override void OnDisconnect (Controller controller)
	{
        //Note: not dispatched when running in a debugger.
		SafeWriteLine ("Disconnected");
	}

	public override void OnExit (Controller controller)
	{
		SafeWriteLine ("Exited");
	}

	public override void OnFrame (Controller controller)
	{
		// Get the most recent frame and report some basic information
		Frame frame = controller.Frame ();

		SafeWriteLine ("Frame id: " + frame.Id
                    + ", timestamp: " + frame.Timestamp
                    + ", hands: " + frame.Hands.Count
                    + ", fingers: " + frame.Fingers.Count
                    + ", tools: " + frame.Tools.Count
                    + ", gestures: " + frame.Gestures ().Count);

		foreach (Hand hand in frame.Hands) {
			SafeWriteLine ("  Hand id: " + hand.Id
						+ ", palm position: " + hand.PalmPosition);
			// Get the hand's normal vector and direction
			Vector normal = hand.PalmNormal;
			Vector direction = hand.Direction;

			// Calculate the hand's pitch, roll, and yaw angles
			SafeWriteLine ("  Hand pitch: " + direction.Pitch * 180.0f / (float)Math.PI + " degrees, "
                        + "roll: " + normal.Roll * 180.0f / (float)Math.PI + " degrees, "
                        + "yaw: " + direction.Yaw * 180.0f / (float)Math.PI + " degrees");

			// Get the Arm bone
            Arm arm = hand.Arm;
			SafeWriteLine ("  Arm direction: " + arm.Direction
                        + ", wrist position: " + arm.WristPosition
                        + ", elbow position: " + arm.ElbowPosition);

			// Get fingers
			foreach (Finger finger in hand.Fingers) {
				SafeWriteLine ("    Finger id: " + finger.Id
					        + ", " + finger.Type.ToString()
					        + ", length: " + finger.Length
					        + "mm, width: " + finger.Width + "mm");

				// Get finger bones
				Bone bone;
				foreach (Bone.BoneType boneType in (Bone.BoneType[]) Enum.GetValues(typeof(Bone.BoneType)))
				{
					bone = finger.Bone(boneType);
					SafeWriteLine("      Bone: " + boneType
						        + ", start: " + bone.PrevJoint
						        + ", end: " + bone.NextJoint
						        + ", direction: " + bone.Direction);
				}
			}

		}

		// Get tools
		foreach (Tool tool in frame.Tools) {
			SafeWriteLine ("  Tool id: " + tool.Id
				        + ", position: " + tool.TipPosition
				        + ", direction " + tool.Direction);
		}

		// Get gestures
		GestureList gestures = frame.Gestures ();
		for (int i = 0; i < gestures.Count; i++) {
			Gesture gesture = gestures [i];

			switch (gesture.Type) {
			case Gesture.GestureType.TYPE_CIRCLE:
				CircleGesture circle = new CircleGesture (gesture);

                // Calculate clock direction using the angle between circle normal and pointable
				String clockwiseness;
				if (circle.Pointable.Direction.AngleTo (circle.Normal) <= Math.PI / 2) {
					//Clockwise if angle is less than 90 degrees
					clockwiseness = "clockwise";
				} else {
					clockwiseness = "counterclockwise";
				}

				float sweptAngle = 0;

                // Calculate angle swept since last frame
				if (circle.State != Gesture.GestureState.STATE_START) {
					CircleGesture previousUpdate = new CircleGesture (controller.Frame (1).Gesture (circle.Id));
					sweptAngle = (circle.Progress - previousUpdate.Progress) * 360;
				}

				SafeWriteLine ("  Circle id: " + circle.Id
                               + ", " + circle.State
                               + ", progress: " + circle.Progress
                               + ", radius: " + circle.Radius
                               + ", angle: " + sweptAngle
                               + ", " + clockwiseness);
				break;
			case Gesture.GestureType.TYPE_SWIPE:
				SwipeGesture swipe = new SwipeGesture (gesture);
				SafeWriteLine ("  Swipe id: " + swipe.Id
                               + ", " + swipe.State
                               + ", position: " + swipe.Position
                               + ", direction: " + swipe.Direction
                               + ", speed: " + swipe.Speed);
				break;
			case Gesture.GestureType.TYPE_KEY_TAP:
				KeyTapGesture keytap = new KeyTapGesture (gesture);
				SafeWriteLine ("  Tap id: " + keytap.Id
                               + ", " + keytap.State
                               + ", position: " + keytap.Position
                               + ", direction: " + keytap.Direction);
				break;
			case Gesture.GestureType.TYPE_SCREEN_TAP:
				ScreenTapGesture screentap = new ScreenTapGesture (gesture);
				SafeWriteLine ("  Tap id: " + screentap.Id
                               + ", " + screentap.State
                               + ", position: " + screentap.Position
                               + ", direction: " + screentap.Direction);
				break;
			default:
				SafeWriteLine ("  Unknown gesture type.");
				break;
			}
		}

		if (!frame.Hands.IsEmpty || !frame.Gestures ().IsEmpty) {
			SafeWriteLine ("");
		}
	}
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

class SampleListener : Listener

{

private Object thisLock = new Object ();

private void SafeWriteLine (String line)

{

lock (thisLock) {

Console.WriteLine (line);

}

public override void OnInit (Controller controller)

{

SafeWriteLine ("Initialized");

}

public override void OnConnect (Controller controller)

{

SafeWriteLine ("Connected");

controller.EnableGesture (Gesture.GestureType.TYPE_CIRCLE);

controller.EnableGesture (Gesture.GestureType.TYPE_KEY_TAP);

controller.EnableGesture (Gesture.GestureType.TYPE_SCREEN_TAP);

controller.EnableGesture (Gesture.GestureType.TYPE_SWIPE);

}

public override void OnDisconnect (Controller controller)

{

//Note: not dispatched when running in a debugger.

SafeWriteLine ("Disconnected");

}

public override void OnExit (Controller controller)

{

SafeWriteLine ("Exited");

}

public override void OnFrame (Controller controller)

{

// Get the most recent frame and report some basic information

Frame frame = controller.Frame ();

SafeWriteLine ("Frame id: " + frame.Id

+ ", timestamp: " + frame.Timestamp

+ ", hands: " + frame.Hands.Count

+ ", fingers: " + frame.Fingers.Count

+ ", tools: " + frame.Tools.Count

+ ", gestures: " + frame.Gestures ().Count);

foreach (Hand hand in frame.Hands) {

SafeWriteLine (" Hand id: " + hand.Id

+ ", palm position: " + hand.PalmPosition);

// Get the hand's normal vector and direction

Vector normal = hand.PalmNormal;

Vector direction = hand.Direction;

// Calculate the hand's pitch, roll, and yaw angles

SafeWriteLine (" Hand pitch: " + direction.Pitch * 180.0f / (float)Math.PI + " degrees, "

+ "roll: " + normal.Roll * 180.0f / (float)Math.PI + " degrees, "

+ "yaw: " + direction.Yaw * 180.0f / (float)Math.PI + " degrees");

// Get the Arm bone

Arm arm = hand.Arm;

SafeWriteLine (" Arm direction: " + arm.Direction

+ ", wrist position: " + arm.WristPosition

+ ", elbow position: " + arm.ElbowPosition);

// Get fingers

foreach (Finger finger in hand.Fingers) {

SafeWriteLine (" Finger id: " + finger.Id

+ ", " + finger.Type.ToString()

+ ", length: " + finger.Length

+ "mm, width: " + finger.Width + "mm");

// Get finger bones

Bone bone;

foreach (Bone.BoneType boneType in (Bone.BoneType[]) Enum.GetValues(typeof(Bone.BoneType)))

{

bone = finger.Bone(boneType);

SafeWriteLine(" Bone: " + boneType

+ ", start: " + bone.PrevJoint

+ ", end: " + bone.NextJoint

+ ", direction: " + bone.Direction);

}

// Get tools

foreach (Tool tool in frame.Tools) {

SafeWriteLine (" Tool id: " + tool.Id

+ ", position: " + tool.TipPosition

+ ", direction " + tool.Direction);

}

// Get gestures

GestureList gestures = frame.Gestures ();

for (int i = 0; i < gestures.Count; i++) {

Gesture gesture = gestures [i];

switch (gesture.Type) {

case Gesture.GestureType.TYPE_CIRCLE:

CircleGesture circle = new CircleGesture (gesture);

// Calculate clock direction using the angle between circle normal and pointable

String clockwiseness;

if (circle.Pointable.Direction.AngleTo (circle.Normal) <= Math.PI / 2) {

//Clockwise if angle is less than 90 degrees

clockwiseness = "clockwise";

} else {

clockwiseness = "counterclockwise";

}

float sweptAngle = 0;

// Calculate angle swept since last frame

if (circle.State != Gesture.GestureState.STATE_START) {

CircleGesture previousUpdate = new CircleGesture (controller.Frame (1).Gesture (circle.Id));

sweptAngle = (circle.Progress - previousUpdate.Progress) * 360;

}

SafeWriteLine (" Circle id: " + circle.Id

+ ", " + circle.State

+ ", progress: " + circle.Progress

+ ", radius: " + circle.Radius

+ ", angle: " + sweptAngle

+ ", " + clockwiseness);

break;

case Gesture.GestureType.TYPE_SWIPE:

SwipeGesture swipe = new SwipeGesture (gesture);

SafeWriteLine (" Swipe id: " + swipe.Id

+ ", " + swipe.State

+ ", position: " + swipe.Position

+ ", direction: " + swipe.Direction

+ ", speed: " + swipe.Speed);

break;

case Gesture.GestureType.TYPE_KEY_TAP:

KeyTapGesture keytap = new KeyTapGesture (gesture);

SafeWriteLine (" Tap id: " + keytap.Id

+ ", " + keytap.State

+ ", position: " + keytap.Position

+ ", direction: " + keytap.Direction);

break;

case Gesture.GestureType.TYPE_SCREEN_TAP:

ScreenTapGesture screentap = new ScreenTapGesture (gesture);

SafeWriteLine (" Tap id: " + screentap.Id

+ ", " + screentap.State

+ ", position: " + screentap.Position

+ ", direction: " + screentap.Direction);

break;

default:

SafeWriteLine (" Unknown gesture type.");

break;

}

if (!frame.Hands.IsEmpty || !frame.Gestures ().IsEmpty) {

SafeWriteLine ("");

}

SampleListener中前几个事件不做说明，要点在于onFrame这个东西。这个东西和OpenCV中Frame概念一致，即“在每一帧上的数据”，关于“Frame”这个东西，我们能获取到以下数据：

Console.WriteLine("Frame id: " + frame.Id
                    + ", timestamp: " + frame.Timestamp
                    + ", hands: " + frame.Hands.Count
                    + ", fingers: " + frame.Fingers.Count
                    + ", tools: " + frame.Tools.Count
                    + ", gestures: " + frame.Gestures ().Count);

Console.WriteLine("Frame id: " + frame.Id

+ ", timestamp: " + frame.Timestamp

+ ", hands: " + frame.Hands.Count

+ ", fingers: " + frame.Fingers.Count

+ ", tools: " + frame.Tools.Count

+ ", gestures: " + frame.Gestures ().Count);

1. Frame ID

2.手个数

3.手指个数

4.工具个数（Leap可以识别出棍子这样的东西）

5.手势个数

这些都是些统计数据，没有什么特别的意义，官方example在Frame中提供了上面五个玩意的详细使用方法。第一个就是手的细节参数。

细读程序，可以发现，手包括了：手心，手方向，手臂，手指，骨头（关节翻译的比较合适一些），图示是官方的一个配置程序，可以看到，Hand包含的所有要素。

手的角度，采用了PitchRowYaw坐标系定义，这样减轻了很多计算负担，只以手自身姿态为参照，确实给开发省去不少事情。手臂提供了关节和手腕的方向。手指提供了手指编号（即手指类型），长度（精度竟然是毫米！可以做为解锁用了）。骨头提供了类型，开始结束（即先后关节），方向。很明显，Leap程序是以人关节位置作为参考点处理的，非常聪明，一般做OpenCV时候，我们只是处理外手形状，找明显分割点来处理的，精度还行，不过极易被外部环境干扰，比如胖子的脖子2333333。以Leap这样处理，精细，稳定，对开发者来说，提供了友好的方式处理手势，为开发提供了方便。

工具，简单，只有ID，position，direction，没有其他东西，够用就好，乔布斯说过，手指是最好的工具，要触摸笔干嘛。

手势，内置了几种手势操作，在激发时候系统能够自动识别的，分别是画圈，横扫，点击，向屏幕点击。在操作时，记得按照官方给的属性操作就好。

Circle — A single finger tracing a circle.
Swipe — A long, linear movement of a finger.
Key Tap — A tapping movement by a finger as if tapping a keyboard key.
Screen Tap — A tapping movement by the finger as if tapping a vertical computer screen.

值得一提的是，Leap提供了一个单独的Touch Emulation，可以模仿平时大家熟悉的手机触摸操作。这点单独拿出来，看来Leap也是动了心思的，识别阈值甚至可以达到毫米级别，当然要这么用，你开心就好。

分析了官方提供的Example后，自己写一个简单的参数获取就异常简单了，今天早上拿到的Leap,下午上课+调试飞机，晚上就将测试程序写了出来，有个几个小点要注意，一是记得引用LeapCSharp.NET3.5.dll或者LeapCSharp.NET4.0.dll，然后添加Leap.dll Leap.lib LeapCSharp.dll到项目中，二是不知为何，需要在application startup文件夹下，把LeapCSharp.NETx.0.dll添加进去，否则实例化时会出错。

简单的Leap就这样创建好了，这几天试试做一个简单的东西来利用Leap的这些数据。

自己实例化的样例：http://download.csdn.net/detail/prius0304/9506700