Multifeature Image and Video Content-Based storage and retrieval

Abstract

In this paper we present most recent evolution of JACOB, a system we developed for image and video content-based storage and retrieval. The system is based on two separate archives: a 'features DB' and a 'raw-data DB'. When a user puts a query, a search is done in the 'features DB'; the selected items are taken form the 'raw-data DB' and shown to the user. Two kinds of sessions are allowed: 'database population' and 'database querying'. During a 'database population' session the user inserts new data into the archive. The input data can consist of digital images or videos. Videos are split into shots and for each shot one or more representative frames are automatically extracted. Shots and r-frames are then characterized, either in automatic or semi-automatic way, and stored in the archives. Automatic features' extraction consist of computing some low-level global features. Semi-automatic features' extraction is done by using annotation tools that perform operations that aren't currently possible with fully automatic methods. To this aim semi-automatic motion based segmentation and labeling tools have been developed. During a 'database querying' session, queries direct or by example are allowed. Queries may be iterated and variously combined to satisfy the query in the smallest number steps. Multifeature querying is based on statistical analysis of the feature space.