[转]XBox One Kinect中的高精度人脸追踪和人脸建模-小明编程

这是微软Kinect人脸建模团队一名开发者的文章，之前硕士我毕设就是受到他们许多启发的。他们很少发文章，一年也就几篇，他们时间可能完全用在算法研究之上。他们这次带来的是最新的人脸建模技术，并且说近期也会发布相关技术文章。（从一代开始，我等了2年，都没见到文章呢。。。）这次的人脸模型真是高精度啊。

关键技术是AAM+基于ICP的3D跟踪，数据使用彩色+深度。说目前在XBox One XDK中才能使用其API（Windows下的API也得等2014夏季了）。视频来自youtube，我无法看到。

High Precision Face Tracking and Face Modelling in XBox One Kinect

Posted by nsmoly on February 14, 2014

My team at Microsoft shipped the high definition face tracking and face shape modelling tech as part of XBox One Kinect back in November 2013. These are some facial expressions that can be tracked by Kinect and animated in 3D with pretty good precision:

The system finds a big number of semantic points on a user’s face in 3D. The developers can then use this information to animate avatars or do some other processing (like recognize expressions, etc.).

This video shows the early prototype of this technology that I made back in 2012:

This is a bit more technical video that demos the 3D mask painted over the video stream (for those who like this stuff):

The algorithm that we created is a fusion of a 2D feature tracker based on Active Appearance Model (AAM) and a 3D tracker based on Iterative Closest Point (ICP) algorithm that aligns 3D face model based on depth data from Kinect. Both trackers are resolved together and linked by special 2D-3D constraints such that the alignment makes sense. We are going to publish details as a paper some time soon.

We also compute 3D face shapes for a given user from a set of input frames (RGB + depth). The resulting 3D models are pretty realistic and look like this: