Motion-based user interaction Write a program to compute a low-resolution motion field in order to interactively control a simple application (Cutler and Turk 1998). For example: 1\. Downsample each image using a pyramid and compute the optical flow (spline-based or pixel-based) from the previous frame. 2\. Segment each training video sequence into different "actions" (e.g., hand moving inwards, moving up, no motion) and "learn" the velocity fields associated with each one. (You can simply find the mean and variance for each motion field or use something more sophisticated, such as a support vector machine (SVM).) 3\. Write a recognizer that finds successive actions of approximately the right duration and hook it up to an interactive application (e.g., a sound generator or a computer game). 4\. Ask your friends to test it out.

Step by step solution

01

Image Downsampling and Optical Flow Calculation

To achieve this, use an image pyramid scheme where images are progressively reduced in size. Downsample each image and compute the optical flow from the previous frame. Optical flow is the pattern of apparent motion of image objects between two consecutive frames caused by the movemement of object or camera. It is computed based on pixel intensity patterns.

02

Segment Video Sequence and Learn Velocity Fields

Segment each training video sequence into different actions i.e., hand moving inwards, moving up, no motion and 'learn' the velocity fields for each one. This essentially means understanding what sort of motion field corresponds to each action. This could be done by finding the mean and variance for each motion field or could use a more complex method such as a support vector machine (SVM).

03

Write a Recognizer

A recognizer needs to be written to detect the successive actions of the right duration and then connect it to an interactive application such as a sound generator or a computer game. This recognizer will use the data prepared in the previous step to understand what action is being taken.

04

Testing

After you’ve completed your recognizer, have it tested. Ask friends to interact with the application through motions and observe if the recognizer is correctly identifying the actions and if the application reacts correctly.

Unlock Step-by-Step Solutions & Ace Your Exams!

• Full Textbook Solutions

  Get detailed explanations and key concepts

• Unlimited Al creation

  Al flashcards, explanations, exams and more...

• Ads-free access

  To over 500 millions flashcards

• Money-back guarantee

  We refund you if you fail your exam.

Start your free trial

Over 30 million students worldwide already upgrade their learning with Vaia!