Friday, November 25, 2011

Improving Finger Detection

The next version of Candescent NUI will find the direction the fingers are pointing at (2D for the moment). I just started working on this but the first tests look promising:
 

I also want to make the code more modular, so it will be possible to "plug in" different algorithms for the various steps without the need to work directly with the source code.


 
 

12 comments:

  1. Amazing. I'm trying to write an AS3 Flash finger tracker. How did you manage to track the fingers so accurately? What is your process?

    ReplyDelete
  2. Hi Kat & Mark

    Thanks! Unfortunately the source code and the information on my blog are currently the only documentation.

    Input data is a depth map (640x480 points with depth information in mm) obtained from the Kinect.

    A short abstract of how it works:
    1. Depth segmentation (fixed max and min depth)
    2. Clustering (k-means) to separate the hands
    3. Find the convex hull (Graham Scan)
    4. Find the contour
    5. Find points where the contour is curved that are also near a point of the convex hull

    Regards
    Stefan

    ReplyDelete
  3. Hi stefan, I havent have time to look into your code yet, but please allow me to ask some simple questions

    1) did you use k-curvature to find the finger tip? is it you the points on the convex hull's corner, then u find the angle?
    2) how do you find the straight line? I was using a simpler approach which is, from the finger tip, start finding the middle point on the left and right of the finger tip point. find around 30-50 points then you will get a series of points representing a straight line. However I m still struggling to find the equation of the line.
    3) what does the orange points at the valley represent? did you use convexity defects concept?
    4) the last but most important question, how to you FIT IN the biggest circle in to the middle of palm? (this if very important for me)
    Thank you very much

    ReplyDelete
  4. Hi TcBoY88

    1&2) I used a simpler algorithm, it's described here: http://blog.candescent.ch/2011/03/finger-tip-detection-part-2.html

    3) The orange points are the "base points" and are used to find the finger direction (orange line). They are a given number of contour points away from the finger tip. The source for this is not on CodePlex yet, I'll describe it in a post later this weekend.

    4) Here's the post about the circle inside the palm: http://blog.candescent.ch/2011/04/center-of-palm-hand-tracking.html

    Basically you find the point that maximizes the distance to all points in the contour.

    I'll sum up the whole algorithm in an article someday, but I can't say how long that'll take.

    Regards
    Stefan

    ReplyDelete
  5. I tried to understand ur code on the circle inside the palm but I failed. lol...
    For now I use another method for it, but it is very unstable.
    I wish to understand ur circle algorithm, but I am just a newbie =(

    My current progres
    http://youtu.be/MmZc1REg6uw

    ReplyDelete
  6. There are some others video of hand tracking in my youtube channel, although the work is not good, but i think the idea is good =)

    hand tracking for gaming, TUIO, google earth etc

    ReplyDelete
  7. Hi TcBoY88

    The algorithm finds the point that maximizes the distance to any point in the contour. Like this you find the circle with the biggest radius that still fits inside the contour.

    I've watched your videos. You're making good progress. I like the blue coloring of the fingers.

    Regards
    Stefan

    ReplyDelete
  8. Hi Stefan thanks for the comment
    What have you been working on recently?

    ReplyDelete
  9. The newest thing I've done was the video manipulation. I haven't published the source for this yet though. And I want to make the whole code more modular.

    ReplyDelete
  10. What about upgrading to use the latest openNI?

    ReplyDelete
  11. Sir
    I am an undergradute student doing my project on hand gesture detection.
    Could please share your code, so that it may help us

    ReplyDelete