Tuesday, April 17, 2012

Candescent NUI - Birthday

Exactly one year ago I've uploaded the first version of Candescent NUI to CodePlex!


Here are some statistics
53'700 page views on this blog
54'400 page views on candescentnui.codeplex.com
10'200 downloads from CodePlex (200 / week)

I've started to look into creating mobile apps. So maybe there'll be posts about Windows Phone development in the future, too.

41 comments:

  1. Happy birthday NUI

    That are impressive numbers!

    ReplyDelete
  2. Happy birthday!
    mobile app with hand/finger tracking using front cam? hehe

    but why windows phone =(

    ReplyDelete
  3. http://channel9.msdn.com/coding4fun/kinect/Revisiting-Candescent-NUI

    you are on channel 9 msdn, great work!

    ReplyDelete
  4. are you working on anything?

    ReplyDelete
  5. Not with the Kinect right now, I'm playing with my Windows Phone.

    ReplyDelete
  6. Awesome! I'm a WP developer too, currently working on Windows 8 apps with MS

    ReplyDelete
  7. Anyone have a complet program written in C# so I can see how this works in context - I need fingertracking data - I bought the Kinect camera and have downloaded the CCT.NUI - but I'm unsure how to use the interface. Thanks for any help here - ToeKnee (o;'

    ReplyDelete
  8. Hi ToeKnee

    You can download the source code and look at the samples.

    - Stefan

    ReplyDelete
  9. http://www.leapmotion.com/
    very precise finger tracking

    ReplyDelete
    Replies
    1. Just saw leap motion today - wow - are you going in this direction Stefan?

      Delete
  10. Stefan

    I keep getting errors in Visual Studion 2010 - got rid of most of them by reinstalling the dll's - I have Kinect SDK Beta 2 - should I be using the first version instead?

    Sorry for all the novice questions - is there some group I can join to better understand OpenNI and all code that uses it with C#?

    Thanks for your guidance
    ToeKnee (o;'

    ReplyDelete
  11. About Leap Motion: The video looks cool, I've registered and tried to apply for the developer kit. But then it was only available in the US, and right now I can't log in. I'll keep an eye on it.

    @ToeKnee:
    You have to use Kinect SDK 1.0, I haven't updated to 1.5 yet.
    There's a google group, but I haven't been active there:
    http://groups.google.com/group/openni-dev?pli=1

    ReplyDelete
  12. hi stefan, did you get any confirmation email from leapmotion for the developer kit application?
    I dont get any email at all. The website is kinda buggy
    The first time I clicked "submit form", it just redirect me back to the homepage

    ReplyDelete
  13. Hi Tcboy

    I did get an e-mail when I had registered an account (only telling me what my password is). But they keep changing their website and the information they want to know when applying for the dev kit. I created an account - now there is no more sign in button. First they only wanted your name and website + description of former projects etc.. now they want a shipping address. It's kind of suspicious..

    I'll wait and see how this develops before giving them more information.

    - Stefan

    ReplyDelete
  14. ya it is suspicious.
    But the developer kit will ship in 1-3months, much faster than pre-orders
    I couldnt resist the temptation of getting 1 earlier

    ReplyDelete
  15. Stefan

    Maybe we can collaborate with Leap Motion - since I am in the US

    Also - I am another person doing sign language - but for my doctorate in CS - I have 2 deaf sisters and a deaf brother - I could use your help on how to capture the data - or basically to understand your code - but as you say - why reinvent the wheel - my work is in the analysis - not the capture - which you've done a beautiful job at. But I definitely need to get your code running. I have MS SDK 1.0 like you said - and it seem that I'm really close - just a few bugs I guess.

    Also - a question for anyone who cares to answer - I keep getting an error message when trying to run handtracking:
    Object Reference not set to an Instance of an Object - anyone had this problem - I'm running on a 64 bit ASUS Notebook G73jh Series

    Thanks for any help here
    ToeKnee Kutscher (o;'

    ReplyDelete
  16. Hi ToeKnee

    Can you send me a stack trace?

    - Stefan

    ReplyDelete
  17. Stefan

    when I start with debugger it stops at the first arrow (see below) then moveds to the next arrow and gives the error message. I'm trying to figure out how to do a stack trace now - thanks for your help - ToeKnee

    namespace CCT.NUI.KinectSDK
    {
    public class SDKDataSourceFactory : IDataSourceFactory
    {
    private KinectSensor sensor;

    => public SDKDataSourceFactory()
    {
    this.sensor = KinectSensor.KinectSensors.First();
    this.Adapter = new KinectSensorAdapter(this.sensor);
    sensor.ColorStream.Enable(ColorImageFormat.RgbResolution640x480Fps30);
    sensor.DepthStream.Enable(DepthImageFormat.Resolution640x480Fps30);
    this.sensor.Start();
    }

    private void radioButtonSDK_CheckedChanged(object sender, EventArgs e)
    {
    Cursor.Current = Cursors.WaitCursor;
    try
    {
    this.dataSourceFactory = new SDKDataSourceFactory();
    }
    catch (Exception exc)
    {
    Cursor.Current = Cursors.Default;
    => MessageBox.Show(exc.Message, "Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
    return;
    }
    this.ToggleButtons();
    Cursor.Current = Cursors.Default;
    }

    ReplyDelete
  18. I just got a messsage (while running the debugger) saying "This feature is not supported by this version of the hardware." Does that mean I need the 32 bit version? - I have a 64 bit machine ASUS Notebook G73jh - if so - when I download - how do I prevent from getting the 64bit version?

    ReplyDelete
  19. Hi ToeKnee

    Do you have a Kinect for Windows or a Kinect for XBOX? This exception is thrown when setting the near mode for Kinect for XBOX (which does not support it).

    If you've got a Kinect for XBOX hardware, I recommend using OpenNI. Kinect SDK with the XBOX hardware does not return any depth values below 800mm, but the algorithm works best at 500 - 800mm.

    - Stefan

    ReplyDelete
  20. Stefan,

    Can you tell me how one would capture all of the depth data from the depthmap within the countour? An example would be great - I need that data to figure out where the fingers and thumb are when the hand is closed. Still waiting for a student loan to get the Kinect for Windows (for better accuracy).

    Thanks for all of your help,
    ToeKnee

    ReplyDelete
  21. Hi ToeKnee

    You can iterate over each line y then move over each point at x in this line while counting how many times you crossed the contour. While the count is not even (1, 3 ...) you add the point (x,y) to the list of points that are inside the contour.

    - Stefan

    ReplyDelete
  22. Thanks Stefan,

    I appreciate all of your help - and - I'm referencing "CandescentNUI" in my paper - do you have some of your own papers I can read/reference for my paper - so that I can give either you or your group credit?

    I modified the Volume Class to do just what you suggested - so I guess I'm on the right track - as I'm getting between 400 and 600 points per hand and have lots of data to look at for my study - Peace - ToeKnee

    public struct Volume
    {
    private IList vPoints;
    private Point location;
    private float width;
    private float height;
    private float depth;
    private float count;

    public Volume(IList points)
    {
    var rangeX = new Range(points.Select(p => p.X));
    var rangeY = new Range(points.Select(p => p.Y));
    var rangeZ = new Range(points.Select(p => p.Z));

    this.vPoints = points;
    this.count = vPoints.Count;
    this.location = new Point(rangeX.Min, rangeY.Min, rangeZ.Min);
    this.width = rangeX.Interval;
    this.height = rangeY.Interval;
    this.depth = rangeZ.Interval;
    }

    public IList Points
    {
    get { return this.vPoints; }
    }

    public float Count
    {
    get { return this.count; }
    }

    ReplyDelete
  23. Hi ToeKnee

    I don't have papers you could reference, but you could put my name in and a link to my website http://www.candescent.ch or the project site:
    http://candescentnui.codeplex.com

    - Stefan

    ReplyDelete
  24. OK then - it will be done
    and thanks so much for all
    of you kind help - talk to
    you in the future.

    ToeKnee (o;'

    ReplyDelete
  25. Hi stefan. Are u planning to release a 32bit version or just have the 64bit?

    Ty

    ReplyDelete
  26. Hi Gamester

    Unfortunately I don't have any 32bit system anymore.

    But you can download the source and replace the DLLs in the lib folder with 32bit versions and then compile it yourself.

    - Stefan

    ReplyDelete
  27. Tried that but didn't working. I'm probably getting the wrong OpenNI 32bit dll anyway. Thanks :)

    ReplyDelete
  28. Hi Stefan,
    U have been playing with WP7 for few months, can u update ur blog w the progress or output?
    Kinda boring for recent months and miss ur blog

    anyway, WP8 SDK is leaked, thought u may be interested

    ReplyDelete
  29. Hi Stefan

    I bought the Kinect for Windows today - changed over to the SDK Version 1 the RGB the Samples all work - but the finger tracker and the clusters run very slow - any suggestions - I had no problems with the Kinect for XBox 360 as long as I used OpenNI - a bit confused - ToeKnee

    ReplyDelete
  30. Hi Stefan

    Forget the last post - I used your newer Kinect SDK V1.5 version and all started working well - I also had some issue with Dropbox taking some CPUs away from me as well - Anyway - thanks again for all you've done - ToeKnee (o;'

    ReplyDelete
  31. This comment has been removed by the author.

    ReplyDelete
  32. Hi TcBoy

    I know, I haven't been very active recently. I was working on my first WP 7 App, but I can't reveal that yet. I'll also have to look into WP 8 programming.

    I'll try to blog more.

    Stefan

    ReplyDelete
  33. Hi ToeKnee

    Ok, cool!

    Sorry for not having answered, I was on holidays in Mongolia for three weeks.

    Stefan

    ReplyDelete
  34. Hi Stefan

    could you show me the C# code to create a new palm center with captured hand-volume data in a method - I was able to create the following:

    private void calculateStaticPalmCenter(ref long[,] theMatrix, ref double palmX, ref double palmY, ref double palmZ)
    {
    List handVolumePoints = new List();
    Core.Point thisPoint = new Core.Point();
    Palm thisPalm;

    for (int row = 0; row < Globals.MAXROW; row++)
    {
    for (int col = 0; col < Globals.MAXCOL; col++)
    {
    thisPoint.Y = (float)row;
    thisPoint.X = (float)col;
    thisPoint.Z = (float)theMatrix[row, col];
    handVolumePoints.Add(thisPoint);
    }

    }

    ConvexHull thisConvexHull = new ConvexHull(handVolumePoints);
    Contour thisContour = new Contour(handVolumePoints);
    //thisPalm = PalmFinder.FindCenter(thisConvexHull, thisContour, handVolumePoints);
    //HandData thisHand = new HandData(1, , thisPalm,handVolumePoints);
    }

    I got this far - But I'm not sure how to create a new palm-center for captured data non-dynamically after I've translated it into a Matrix

    Thanks for your time and expertise
    ToeKnee (o;'

    ReplyDelete
  35. Hi Stefan

    I've captured volume-data points - now I want to create a new palm-center-point - how would I code this in C# here's what I have so far:

    private void calculateStaticPalmCenter(ref long[,] theMatrix, ref double palmX, ref double palmY, ref double palmZ)
    {
    List handVolumePoints = new List();
    Core.Point thisPoint = new Core.Point();
    Palm thisPalm;

    for (int row = 0; row < Globals.MAXROW; row++)
    {
    for (int col = 0; col < Globals.MAXCOL; col++)
    {
    thisPoint.Y = (float)row;
    thisPoint.X = (float)col;
    thisPoint.Z = (float)theMatrix[row, col];
    handVolumePoints.Add(thisPoint);
    }

    }

    ConvexHull thisConvexHull = new ConvexHull(handVolumePoints);
    Contour thisContour = new Contour(handVolumePoints);
    //thisPalm = PalmFinder.FindCenter(thisConvexHull, thisContour, handVolumePoints);
    //HandData thisHand = new HandData(1, , thisPalm,handVolumePoints);
    }

    Please Advise
    ToeKnee (o;'

    ReplyDelete
  36. Hi ToeKnee

    You need to use the class ContourFactory to create a Contour and GrahamScan to create a ConvexHull.

    - Stefan

    ReplyDelete
  37. Hi Stefan

    I'm almost there - but I'm about 3 rows off-center with the current code below - could you advise me as to what I may be doing wrong?

    private void captureStaticPalmCenter(HandData thisHand, long[,] theMatrix, List thePoints,
    ref Core.Point palmPoint, long xMin, long yMin)
    {
    GrahamScan grahamScan = new GrahamScan(thePoints);
    ConvexHull convexHull = grahamScan.FindHull();

    ContourFactory contourFactory = new ContourFactory(5f);
    int[,] intMatrix = new int[50, 50];
    copyLongMatrixToInt(theMatrix, ref intMatrix);
    DepthMap zDepthMap = new DepthMap(intMatrix);
    //Contour thisContour = new Contour();
    Contour contour = contourFactory.CreateContour(zDepthMap, (float)xMin, (float)yMin);

    HandDataSourceSettings handSrcSettings = new HandDataSourceSettings();
    PalmFinder palmfinder = new PalmFinder(handSrcSettings);

    Palm thatPalm = palmfinder.FindCenter(convexHull, contour, thePoints);
    Palm thisPalm = new Palm(thatPalm.Location, thatPalm.DistanceToContour);

    double thisX = thisPalm.Location.X;
    double thisY = thisPalm.Location.Y;
    adjustToMatrix(ref thisX, ref thisY);
    thatPalm.setLocation((float)thisX, (float)thisY);

    palmPoint = thisPalm.Location;
    }

    Thanks for any Help you have time to provide
    ToeKnee (o;'

    ReplyDelete
  38. Hi ToeKnee

    What does adjustToMatrix do?

    Can you send me a running sample?

    - Stefan

    ReplyDelete
  39. Hi Stefan - do you have a personal email I can send the code to?
    Here's the answers to your questions:
    Here's the code - it just makes all points either 5's or 10's to show up in the matrix - but maybe that's the problem?

    Also - left and top parameters confuse me - I gave left the xMin point and top the yMin point - any suggestions?



    void adjustToMatrix(ref double theX, ref double theY)
    {
    if (theX % 5 != 0)
    {
    double xd = theX / 10.0;
    double xf = Math.Floor(xd);
    double xc = Math.Ceiling(xd);
    double x2 = xf + 0.2, x8 = xf + 0.8;
    if (xd >= x2 && xd < x8)
    theX = ((xf + 0.5) * 10);
    else
    if (xd < x2)
    theX = xf * 10;
    else
    theX = xc * 10;
    }
    if (theY % 5 != 0)
    {
    double yd = theY / 10.0;
    double yf = Math.Floor(yd);
    double yc = Math.Ceiling(yd);
    double y2 = yf + 0.2, y8 = yf + 0.8;
    if (yd >= y2 && yd < y8)
    theY = ((yf + 0.5) * 10);
    else
    if (yd < y2)
    theY = yf * 10;
    else
    theY = yc * 10;
    }
    }

    ReplyDelete
  40. You can use info@candescent.ch

    ReplyDelete