Sunday, April 3, 2011

Performance

While refactoring the clustering and hand detection code, I noticed a drop in the framerate. I didn't add any new functionality, so I did some profiling (I'm lacking a decent .Net Profiler and won't pay 400 - 600 USD for one in a private project). Here is what I found:

I did add an additional method call inside the loop over the depth values to filter the value instead of putting the expression directly into an if. And I did move the width and height of the map into a Size field and used this as the boundary in the two for loops, plus I introduced a ClusteringSettings class where I placed some boundary values.
  1. if(IsValidValue(...)) instead of if(depthValue > minDepth ...)
  2. for(int x = 0; x < this.size.Width; x++) instead of for(int x = 0; x < width; x++)
  3. this.settings.MinDepth instead of ushort minDepth = ...
The following times were measured on my Intel Core i7-860:
  • Duration of the clustering after the refactoring: 8.9 ms
  • With local settings (undo point 3): 6.9 ms
  • With local width and height (undo point 2): 5.2 ms
  • Expression directly in the if (undo point 1): 3.8 ms
This means the refactoring had more than doubled the required time to iterate over the depth values. I had not payed enough attention to performance.

Kinects depth camera works with VGA resolution (640x480) at 30 frames per second, which means you have to process 9'216'000 depth values per seconds. It also means that processing must not take longer than 33.3... milliseconds per frame. Otherwise, because new frames come faster than you process them, you have to skip some frames to keep up.

The hand and finger detection currently takes around 12 ms per hand per frame. That's 24 milliseconds for both hands, plus the 8.9 milliseconds for the clustering adds to around 33 milliseconds. And that is where the frames start to get lost.

Of course the next task is to optimize the hand and finger detection code.

4 comments:

  1. Hi radu

    Short answer: Currently not

    Long answer: I plan to release a .NET class library (free for personal use) which other applications can use to receive hand and finger tracking events.

    I don't know yet if I want to publish the source code too. I also can't announce a date, because I don't know how much time I can find beside my regular job during the next few weeks / months.

    Regards,
    Stefan

    ReplyDelete
  2. hi Stefan
    can you share your code about fingertip

    ReplyDelete
  3. The source code is available at:
    candescentnui.codeplex.com

    ReplyDelete