Saturday, June 18, 2011

Candescent NUI (7665)

I've put a new version on CodePlex (source and binary) and changed the official release to this version.

Please read the new quick start, because the creation of data sources has changed! http://candescentnui.codeplex.com/documentation

For those who use Candescent NUI and want to try the SDK: I'd recommend against installing it at the moment. For this project there are no advantages, only disadvantages. For example the SDK returns no depth data closer than 85cm. I haven't compared performance yet.

There are advantages over OpenNI in general:
- Easier installation
- Audio data is supported
- You can control the motor
- Personally I like that the skeleton tracking does not need the Y calibration pose

14 comments:

  1. I tested using the Kinect SDK, and it seems that the depth image is inverted.

    ReplyDelete
  2. You're right.. I didn't see that, oops :-)

    In OpenNI there is an option to mirror the image. I've corrected that issue in change set 7665.

    (I'll edit this post to point to the new release).

    ReplyDelete
  3. hey stefan,

    i am wondering if i can get the handdata and depth data at the same time... is that possible?
    because i want to get (your) handdata from my left hand and do some other stuff with the right hand, where depth data is needed for me...

    (anonymous) ;)

    ReplyDelete
  4. Hi Bernhard

    Yes it's possible, but you have to set up your data sources manually, like this:

    private IHandDataSource handDataSource;
    private IDepthPointerDataSource depthPointerDataSource;

    private void button1_Click(object sender, EventArgs e)
    {
    var openNIDataSourceFactory = new OpenNIDataSourceFactory("config.xml");
    depthPointerDataSource = new DepthPointerDataSource(openNIDataSourceFactory.GetDepthGenerator());
    handDataSource = new HandDataSource(new ClusterDataSource(depthPointerDataSource));
    handDataSource.NewDataAvailable += new NewDataHandler(handDataSource_NewDataAvailable);
    handDataSource.Start();
    }

    void handDataSource_NewDataAvailable(HandCollection data)
    {
    var depthData = this.depthPointerDataSource.CurrentValue; //Pointer to the depth data
    }

    Regards,
    Stefan

    ReplyDelete
  5. Since it's still a Beta, you should talk to Microsoft. eventally we'll all need to move to the SDK and I love your project - I wouldn't want to leave it behind.

    ReplyDelete
  6. Besides from the SDK you should try to generate touch point events in WPF with your fingers at a certain depths to automatically let windows handle multitouch with WPF to make it behave like the Surface.

    ReplyDelete
  7. Hi EddHead

    Yes, I guess I should ask Microsoft if they plan to return depth values closer than 85cm.

    I'll try to improve the algorithm to work better around 85cm to 1 meter. And there is also the possibility to use a better resolution than 640x480, which will enable finger detection farther away (but also make it harder because a lot more points have to be processed).

    On the other hand I've heart rumours that MS themselves will provide finger detection somewhen in the future. I guess that'll be the point when I have to bring something new or let the project end.

    Regards,
    Stefan

    ReplyDelete
  8. you could also use performance enhancements like process areas of the image only around the hands so that you don't need to process the whole image - maybe that can improve the performance tenfold. since you get access to the body without the calibration - so it makes it more relevant.

    ReplyDelete
  9. I've asked about the min distance here:

    http://social.msdn.microsoft.com/Forums/en/kinectsdknuiapi/thread/53a62e18-6acf-466b-9549-9692ff183d77

    ReplyDelete
  10. nice, in any case - what about moving the hand calculations to the GPU to save processing time? have you tried GPU programming before?

    ReplyDelete
  11. Accelerator .NET library seems to be a good start for this.

    ReplyDelete
  12. I have no experience in GPU programming. I'll first try the Task Parallel Library but when I find time I'll have a look at Accelerator.NET.

    ReplyDelete
  13. Really awesome project, works well i'm looking forward to it.

    The solution for "I tested using the Kinect SDK, and it seems that the depth image is inverted.":
    change the line 67 in MSClusterDataSource.cs
    for (int x = 0; x < localWidth; x++)
    to
    for (int x = localWidth; x > 0; x--)

    And the depth image should show correctly.
    Hope this help.

    ReplyDelete
  14. Hi Deacon

    Thanks for the hint. I had corrected that already one time in changeset 7665, but the change got lost in changeset 8120.

    I guess I had forgotten to also apply this fix in my private repository. I'm currently developing with a seperate repository where I can check things in that don't automatically show on CodePlex. When I have enough (or urgent) changes I put it on CodePlex.

    I've reapplied the changes in 8263.

    Regards,
    Stefan

    ReplyDelete