MultiMedia Visual Information Seeking

Stacie Lynn Hibino
University of Michigan


Table of Contents


Dedication

Acknowledgments

Preface (see Abstract)

List of Figures

List of Tables

Chapter

  1. Introduction
    2.
    1.1 Motivation
    1.2 Previous Research
    1.3 Problem Statement
    1.4 Research Goals
    1.5 Executive Summary
    1.6 Contributions
    1.7 Overview

  2. Overall Framework and Annotation Model

    3. 2.1 Introduction
    2.2 Overall Framework
    2.2.1 Overview of Framework
    2.2.2 User Interface Components
    2.2.3 System Components
    2.3 Annotation Model
    2.4 Specifying Relative Queries
    2.5 Chapter Summary

  3. TVQL: A Temporal Visual Query Language

    4. 3.1 Introduction
    3.2 The Relative Duration Query Filter
    3.3 Specifying Relative Temporal Position
    3.3.1 Primitive Temporal Relationships
    3.3.2 Constraints and Interdependencies Between Filters
    3.3.3 Combining Temporal Primitives by Neighborhood
    3.4 Disjunctive Operator: Combining Temporal Primitives and Neighborhoods
    3.5 User-Created Macros
    3.6 Visual Enhancements to TVQL
    3.6.1 Temporal Diagrams
    3.6.2 Descriptive Labels: Additional Textual and Visual Aids
    3.7 Combining Relative Temporal Duration and Position Query Filters
    3.8 Comparing TVQL to a Text-Based Temporal Query Language
    3.8.1 Advantages of TVQL
    3.8.2 Limitations of TVQL
    3.9 Chapter Summary

  4. TViz: Temporal Visualization of Results

    5. 4.1 Introduction
    4.2 The Main MMVIS Window
    4.3 Visualization of the Selected Event Subsets
    4.4 Visualization of the Temporal Relationships
    4.5 Chapter Summary

  5. Processing TVQL Queries

    6. 5.1 Introduction
    5.2 Background on Processing TVQL Queries
    5.2.1 Basic DQ Characteristics
    5.2.2 TVQL Parameters
    5.2.3 TVQL Query Processing
    5.3 Index Structures for Multidimensional Range Queries
    5.3.1 The k-Array and k-Bucket Index Structures
    5.3.2 Other Indexes for Processing Multidimensional Range Queries
    5.3.3 Discussion of Index Structures for Multidimensional Range Queries
    5.4 Experimental Design
    5.4.1 Assumptions
    5.4.2 Simulation Environment
    5.4.3 Query Descriptions
    5.4.4 Query Rationale
    5.4.5 Sample Scenario
    5.5 Experimental Results
    5.5.1 Clustering the Data
    5.5.2 Incremental vs. Non-incremental Queries
    5.5.3 Evaluating the Array-Based Methods for Incremental Processing
    5.5.4 Comparing Index Methods for Incremental Processing
    5.6 Cost to Build Index Structures
    5.7 Chapter Summary

  6. A Case Study Using MMVIS for Temporal Analysis of Real CSCW Video Data

    7. 6.1 Introduction to the Case Study
    6.1.1 Description of the Data Set
    6.1.2 Chapter Overview
    6.2 Using Subset Selection Visualizations as Part of the Analysis Process
    6.2.1 Selecting A and B Subsets
    6.2.2 Comparing Relative Frequency of Selected Events
    6.2.3 Comparing Relative Average Duration of Selected Events
    6.2.4 Comparing Relative Total Duration of Selected Events
    6.2.5 Comparing Frequency, Average and Total Duration of Selected Events
    6.3 Using Relationship Visualizations for Temporal Analysis
    6.3.1 Examining the All Starts and Initiates Temporal Relationships
    6.3.2 Examining the Ends Temporal Relationship
    6.3.3 Examining the Context Switching Temporal Relationship
    6.4 Chapter Summary

  7. Evaluating the TVQL User Interface

    8. 7.1 Introduction
    7.2 A Forms-Based Temporal Query Language (TForms)
    7.3 Experimental Design
    7.3.1 Participants
    7.3.2 Design
    7.3.3 Procedure and Materials
    7.3.4 Types of Data Collected
    7.4 Results and Discussion
    7.4.1 Efficiency Evaluation
    7.4.2 Accuracy Evaluation
    7.4.3 User Satisfaction Evaluation
    7.5 Proposed Changes to TVQL
    7.6 Chapter Summary

  8. Evaluating the MMVIS Framework for Temporal Analysis

    9. 8.1 Introduction
    8.2 Experimental Design
    8.2.1 Participants
    8.2.2 Design
    8.2.3 Procedure and Materials
    8.2.4 Types of Data Collected
    8.3 Evaluating the MMVIS User Interface
    8.3.1 Subset Selection
    8.3.2 TViz
    8.3.3 TVQL
    8.3.4 MMVIS User Satisfaction Ratings
    8.4 Evaluating the Timeline User Interface
    8.4.1 Timeline Layout
    8.4.2 Rescaling Timelines
    8.4.3 Timeline Summary Information
    8.4.4 Annotating Timelines
    8.4.5 Summary of Timeline Evaluation
    8.5 Comparing MMVIS to the Timeline for Analysis of Temporal Trends
    8.5.1 Participants
    8.5.2 Data Analyzed
    8.5.3 Coding Scheme and Overall Results
    8.5.4 Analyzing the Temporal Occurrences and Duration of Events
    8.5.5 Analyzing Trends in Temporal Relationships Between Various Events
    8.5.6 Comparing User Satisfaction
    8.5.7 Incorporating a Timeline into MMVIS
    8.6 Chapter Summary and Future Work

  9. Related Work

    10. 9.1 Bit-Level Video Analysis
    9.2 Video Annotation and Analysis
    9.3 Multimedia Databases
    9.4 Direct Manipulation Query Languages
    9.5 Temporal Query Languages
    9.6 Dynamic Queries and Visual Information Seeking
    9.7 Processing Multidimensional Range Queries

  10. Conclusion and Future Directions

    11. 10.1 Contributions
    10.2 Future Directions
    10.2.1 Incorporation and Enhancements of Supporting MMVIS Components
    10.2.2 Preliminary Work on a Direct Manipulation Spatial Query Paradigm
    10.2.3 Preliminary Work on a Direct Manipulation Motion Query Paradigm

Appendices

Bibliography

Intro | Work & Research Activities | Papers & Honors
Education | Activities & Family
last updated 02/10/2018, hibino at acm dot org