Saturday, December 24, 2011

Candescent NUI 11936

I've created a new release of Candescent NUI. It can be downloaded here: [source] or [binary]

Here's a list with the changes.

HandDataSource Split
I've moved part of the code that was in CCT.NUI.HandTracking to the Core library. There's a new namespace CCU.NUI.Core.Shape that contains a new kind of data source: IShapeDataSource

 I've moved the convex hull and contour detection to this new data source. The process is now:
1. Clustering: Creates clusters
2. Shape processing: Creates shapes that have a convex hull and a contour
3. Hand detection: Uses shapes to detect hand data (like finger tips and center of palm)

There's a new class in town: HandDataFactory
When you want to control the setup and handling of OpenNI or the Kinect SDK yourself and not use the DataSource classes as required until now, you can create a HandDataFactory. It offers only two methods, one that creates hand data from a pointer to OpenNI depth data and the other that takes a ImageFrame returned by the KinectSDK.

var factory = new HandDataFactory();

//OpenNI
var handData = factory.Create(depthGenerator.DataPtr);

//Kinect SDK
var handData = factory.Create(imageFrame);

Other changes
- Contour points now also have a Z value
- Performance should be a bit better
- Internal refactoring

13 comments:

  1. Sometimes when pressing the RGB button, or the hand & finger detection button I'm getting a NullReferenceException in the ImageGeneratorAdapter on line 22, stating that this.Generator is null. Do you have any what could cause this problem and maybe how to fix it?

    I'm using the latest OpenNI unstable.

    ReplyDelete
  2. Hi Anonymous

    I've seen that happen too, but only rarely.

    I think this can only happen when the OpenNI context returns null or something that is not an ImageGenerator (in OpenNIDataSourceFactory.cs):

    return this.context.FindExistingNode(NodeType.Image) as ImageGenerator;

    It didn't happen with the stable version as far as I remember.

    Regards
    Stefan

    ReplyDelete
  3. Hello Stefan,

    It does look like it returns null, so I'll try again with the stable version soon.

    Thanks for the quick response!

    ReplyDelete
  4. Hi, Stefan

    According this article, I have some questions.

    Q1:The hand tracking algorithm and the algorithm of previous version

    are the same, right? Or you have change the algorithm?

    Q2:I have read your source code, and I confuse with that I just find

    hand detection part(cluster, convex hull, finger detection), but I can't

    find how did track the hand? Or the previous is hand tracking part?

    Q3:I can't open the WPF samples, but I can open the others samples, do

    you know the reason?

    Thanks!

    Berry

    ReplyDelete
  5. Hi Berry

    Q1: The algorithm didn't change, just how the code is organized.

    Q2: You're right.. the algorithm does not really track the hand. It just runs the detection for each frame. The only thing that comes close to tracking is that it maps the ids of the hands in the old frame to the ones detected in the new frame (if possible).

    Q3: What's the exception?

    - Stefan

    ReplyDelete
  6. Thanks for your explain in detail, I can know this part of story.

    Q3:I have checked that the exception, and the error code is

    0x8007007E. It seems that it can't load "OpenNI.dll", but when I looked

    for this dll file, I can find it! This situation confused me, do you

    know the reason? Thanks!

    Berry

    ReplyDelete
  7. In what version is it happening? Just 11936, or also in the newest release?

    ReplyDelete
  8. I just try this "11336" project, because begin

    this version, WF and WPF sample are separable.

    Thanks! Berry

    ReplyDelete
  9. When I download the source for 11336 I get two errors (2 test files seem to be missing). But when I exclude the Test project the WPF samples build and I can also run them.

    Maybe it's a problem with 32bit / 64bit OpenNI versions on your machine?

    ReplyDelete
  10. Test project? When I open 11336 project, Ididn't

    see that. By the way, if I use the Kinect SDK to

    develop my application, should I have to install

    OpenNI? According your project, user

    can choose one of them. Or just I

    misunderstand?

    Thanks! Berry

    ReplyDelete
  11. I see.. Only the normal samples project is set up to support both OpenNI and KinectSDK. The WPF samples default to OpenNI.

    Currently I still recommend OpenNI, because the XBOX Kinect doesn't feature the near mode. I still could not obtain the new Kinect For Windows hardware.

    ReplyDelete
  12. This comment has been removed by the author.

    ReplyDelete
  13. I see, I'll try to know all of them.

    Thanks! Berry

    ReplyDelete