Recently I was looking into a Kinect idea for work and needed to re-project the depth map into a point cloud that we could further process. I found it is hard to get information on how to do this properly with the data given by the official SDK, so I thought I would put this up here for future reference.

I found that to get the correct X/Y/Z values, you need to make use of a constant defined in the C++ SDK, called the Inverse Focal Length, which is defined in pixels for our purposes.

float invFocalLength = 3.501e-3f;

Once you have this, you need to go through every pixel in the depth map, and first divide the depth value by 1000 to take it from mm to meters. Then you use the following expressions to get your X and Y values:

z = inputZ / 1000.0f;
outputX = (inputX - 0.5f) * (invFocalLength * z);
outputY = (0.5f - inputY) * (invFocalLength * z);

If you want to use this directly in a 3D scene, it is recommended that you multiply the resulting values by 320 and 240 for X and Y respectively; if you just want to process it, you should leave the data as is.

Note that the X and Y input values are normalised in the range [0, 1] before being used in that equation.