[RealSense SDK] Calibrate Color and Depth Stream

Recently, I play around with RealSense technology using a newly bought SR300 (front-facing 3D camera). I want to stream color and depth stream together, then create an depth stream mask over the color stream. This is similar to what is already been done by 3D Segmentation algorithm of the RS-SDK.

Retrieving color and depth stream at once has been a problem which I have addressed in other post. However, after being able to stream color and depth all together, there is another problem. The color and depth image is not in the same coordinate, even if the stream resolution is the same. This is due to the fact that the physical location of IR sensor is different with the color sensor. As the result, I need to somehow translates either color or depth image so that they fit to one another.

I want to fit the depth and color image so that they are well aligned without needing to write a lot of codes. And, I believe RS-SDK has been equipped with such functionality. Otherwise, it would be silly SDK.

Shortly, after reading the documentation I found something which I was looking for. That is the ‘Projection’ interface. It provides several functions which are:

  1. Mapping color and depth image coordinates.
  2. Projecting color and depth image coordinates, as well as world coordinates.
  3. Creating an aligned (in terms of size and spatial location) color and depth images.

Now that I found that the 3rd functionality is the one that I need, I need to find a specific method which does so. And finally, I ended up with the following methods:

  • CreateColorImageMappedToDepth
  • CreateDepthImageMappedToColor

By using one of the above methods, I can create a depth mask on a color image. Here an example piece of codes which I used to create it:

private void CalibrateColorAndDepthStream(PXCMCapture.Device device, PXCMImage depth, PXCMImage color, out PXCMImage output)
PXCMProjection projection = device.CreateProjection();
output = projection.CreateDepthImageMappedToColor(depth, color);

The output of the above method is a depth image which has been calibrated. So that I can create a 1:1 pixel mapping of color and depth image.


3 thoughts on “[RealSense SDK] Calibrate Color and Depth Stream

  1. Hi~ I use CreateDepthImageMappedToColor function. But the compiler says that it’s a virtual function and I can’t use it. I am very confused. Could you please share the full code of calibrate color and depth stream?Thank you so much!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s