SEVA: Sensor-Enhanced Video Annotation

Publication Date

2009

Journal or Book Title

ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS

Abstract

In this paper, we study how a sensor-rich world can be exploited by digital recording devices such as cameras and camcorders to im-prove a user's ability to search through a large repository of image and video files. We design and implement a digital recording sys-tem that records identities and locations of objects (as advertised by their sensors) along with visual images (as recorded by a camera). The process, which we refer to as sensor-enhanced video annota-tion (SEVA), combines a series of correlation, interpolation, and ex-trapolation techniques. It produces a tagged stream that later can be used to efficiently search for videos or frames containing particular objects or people. We present detailed experiments with a proto-type of our system using both stationary and mobile objects as well as GPS and ultrasound. Our experiments show that: (i) SEVA has zero error rates for static objects, except very close to the boundary of the viewable area; (ii) for moving objects or a moving camera, SEVA only misses objects leaving or entering the viewable area by 1-2 frames; (iii) SEVA can scale to 10 fast moving objects using current sensor technology; and (iv) SEVA runs online using rela-tively inexpensive hardware.

DOI

http://dx.doi.org/10.1145/1556134.1556141

Pages

-

Volume

5

Issue

3

This document is currently not available here.

Share

COinS