In this thesis, I present a new paradigm for the analysis of spatio-temporal data--one that seamlessly integrates querying and browsing of spatio-temporal relationships through an integrated MultiMedia Visual Information Seeking (MMVIS) environment. I designed MMVIS as an interactive visualization framework where users can explore spatio-temporal relationships between subsets of events using a visual query language tightly coupled to a dynamically updated visualization of results. I then applied this approach to the temporal analysis of video data. Key contributions of this research include the definition of a temporal visual query language (TVQL) and corresponding temporal visualization (TViz) of results, the development of the k-Bucket array-based indexing structure for incrementally processing multidimensional range queries, and the evaluation of the efficiency, viability, and usability of the MMVIS approach.
TVQL is a novel query interface that supports specification of temporal queries via direct manipulation and facilitates temporal browsing in a way that was not previously possible. The power of TVQL is enhanced with its tight integration with TViz. TViz visually depicts temporal relationships between subsets by abstracting relationships between individual event instances into trends.
The k-Bucket is a new index structure that I developed for the open problem of processing incremental multidimensional range queries, such as those posed using TVQL. I present the validation of k-Bucket efficiency over existing index structures in a series of simulations. I demonstrate the viability and usability of MMVIS through prototype implementation and through two user studies and a case study applying MMVIS to the analysis of real video data. I compare TVQL to a forms-based query language (TForms) in the first user study, showing that while TVQL takes longer to learn, TVQL subjects are more efficient and more accurate in specifying queries than TForms subjects. The second study compares the utility of MMVIS to a timeline visualization for finding temporal trends, showing that subjects can use either interface to find interesting and complex temporal trends, but that MMVIS subjects are more accurate and are able to detect trends and exceptions, whereas timeline subjects are biased against finding exceptions to trends.