Interactive Multimedia Documents:
Modeling, Authoring and
Implementation Experiences

M. Vazirgiannis
Springer-Verlag, LNCS 1564
October 1999

    The need for multimedia information systems grows rapidly in a variety of fields including business, manufacturing, education, CAD, CAE, medicine etc. Due to the diverse nature of multimedia data, systems designed to store, transport display and  manage such data must have considerably more functionality and capability than conventional information management systems. Multimedia applications can be very complex as regards the number of involved objects, transformations of the objects in the scope of an application and relationships among them. These features become even more salient with the advent and wide usage of the WWW as the main medium of information dissemination. 

    The issue of modeling, authoring, querying and rendering Interactive Multimedia Documents (IMDs) has attracted significant research and industrial efforts in the last years. In this book we present an integrated framework for IMDs, results of our research work in the topic for several years. The book aims at postrgaduate students and research scientists and practitioners and the topics covered include: 

    Multimedia document models & standards
    A review of the research efforts aiming at modeling the various aspects of multimedia documents is provided as well as a substantial presentation of current document standards (HYTIME, MHEG and SMIL). 

    Modeling of IMDs 
    The basis of the framework is a model for IMDs that covers widely the issue of interaction and spatiotemporal composition. According to the model an IDM is defined in terms of actors, events and scenario tuples. The actors (video, sound, image, text, and buttons) represent the media objects participating and their spatiotemporal transformations in order to align to the presentation requirements. The events are the interaction primitives and they may be atomic or complex. They are generated by user actions, actorsí state changes or by the system. The basic constituent of an IMD is the scenario, namely a set of scenario tuples. A tuple is a fundamental entity of functionality in the scenario conveying information about the event(s) that start (or stop) a set of synchronized media presentations. 

    Authoring & verification of IMDs
    Based on this IMD model we develop an authoring methodology, clearly identifying the stages of interaction and spatiotemporal presentation specification. During the authoring process it is vital to provide to the author verification facilities so as to review the document design and thus avoid errors and improve the design quality. The verification has to deal with the temporal and spatial dependencies and constraints of the document. We propose a methodology for verification of the spatiotemporal content of an IMD. A related issue is the ability of an author to query an IMD as regards the spatiotemporal dependencies that are inherent to it. There the requirement for efficiency is apparent. We have worked on a spatiotemporal indexing scheme that processes efficiently queries with intensive spatiotemporal content. 

    Indexing IMDs
    We present a scheme for managing IMDs that involve a large number of interrelated objects. We propose an indexing scheme based on spatial indexing techniques. Thus fast retrieval of queries regarding spatiotemporal queries is feasible. Moreover spatial or temporal application layouts may be obtained. 

    Rendering IMDs
    The result of IMD specification under this approach is a declarative script, which has to be executed whenever and IMD session is initiated. The execution (rendering) of an IMD may be a complicated task due the multitude of occurring events and the potentially large sets of different presentation options. We developed two approaches for IMD rendering, one that is based on a single-thread approach and another generic multi-threaded architecture that detects and evaluates events that occur in the presentation context and triggers the appropriate synchronized presentation actions. Both schemes are implemented. The second presentes substantial potential for WWW exploitation.