Saturday, November 26, 2011

Video Manipulation Interface

I was heavily impressed by this demonstration of a Minority Report inspired image manipulation interface by Garratt Gallagher (for those wo haven't seen the movie here's the scene that shows the interface).


Today I'm trying to take it to the next level. Here's my version of such an interface that focuses on videos. It lets you place (predefined) videos in 3D space on top of the RGB image captured by the Kinect. A simple gesture lets you time-shift the video and another closes the video.


Written in c# using Kinect, OpenNI and WPF4 (this is the very first version and there's a lot to improve...)


These are the input gestures that are currently working

  • Defining a video space with 4 fingers (2 on each hand) starts a new video inside that space
  • Opening the right hand while pointing at a video with one finger of the left hand closes the video
  • Opening the left hand puts the video into time-shift mode. Moving the right finger on the x-axis forwards or rewinds the video

The source code for the video manipulation is not online yet. The hand and finger detection can be downloaded from CodePlex: http://candescentnui.codeplex.com

11 comments:

  1. Wow, briliant ...
    i would like to have alook in your code ...

    greetz
    Mil

    ReplyDelete
  2. Hi Anonymous

    It will take a while until I'll release the code for the video editing. But you can already look at the other samples and the hand and finger detection code on CodePlex:

    http://candescentnui.codeplex.com

    Regards
    Stefan

    ReplyDelete
  3. Genius. i'm so jealous. been too busy with the Windows Phone :(

    ReplyDelete
  4. recently openni upgraded to version 1.4.0.2
    avin2 sensor also has been updated
    Stefan if you have spare time can you also upgrade your program to use the latest version of openni?

    Thank you very much

    TcBoY

    ReplyDelete
  5. @Jay
    Thanks :-)

    Actually I also want to look into Windows Phone programming but have found no time yet.

    ReplyDelete
  6. Hi TcBoY

    I'll publish a new version of Candescent NUI this weekend. For the version after that I'll try to use the new OpenNI version.

    Regards
    Stefan

    ReplyDelete
  7. Excellent stuff, but curious to know what hardware your use in your video. I'm using the Xtion pro live and feel like there is lag. Have you used the asus product? Thanks

    Excellent work.

    ReplyDelete
  8. Hi ITGuy

    I've used the Kinect for XBOX for this video. But it's more likely that your PC produces the lag.

    Stefan

    ReplyDelete
  9. Hi, i test the example but say me that i got a problem with the path of video, i hope to answer me

    ReplyDelete
  10. Hi Alejandra

    You have to change the paths to your own videos. They are not included in the sample (because of their size).

    - Stefan

    ReplyDelete
  11. For the curious: I believe the *complete* code has in fact been added now, including the video manipulation. (You will, however, need to supply your own videos.) It's on http://candescentnui.codeplex.com/ as stated here, but specifically I believe it's in the source tree in the CCT.NUI.Visual package.

    ReplyDelete