Visual IR
Audio Based IR
SEG Scheme
SBD Scheme
Progressive Query
Video Summarization
Hierarchical Cellular Tree
TLQ System
Mobile MUVIS
    Spatial Segmentation (SEG) Framework
    • Similar to FeX framework, a SEGmentation (SEG) framework is designed to dynamically integrate any spatial segmentation algorithm into MUVIS using a dedicated SEG API, which is described in this section. SEG modules are used to create Segmentation Masks (SMs) from an image or key-frame of a video clip. Each SM contains two or more segmented regions (segments) indicated by a distinct 8 bits gray-scale value (between 0-255). Therefore, as a practical limit there can be maximum 256 segments in a SM image. Any SEG module is responsible to assign different (and unique) gray-scale values to the pixels within each segment.
    • SEG API provides the necessary handshaking and information flow between a MUVIS application and the SEG DLL. ”). SEG modules are mainly developed for and used by DbsEditor application during the indexing of the video clips and images in a MUVIS database All the SEG algorithms should be implemented as a Dynamic Link Library (DLL) with respect to the specific SEG API, and stored in an appropriate folder (in the same directory with the applications or in “C:\MUVIS\” directory). The naming convention of a SEG module must be as follows:
      SEG_[fourCC code].dll (i.e. “SEG_RSST.dll”)
      SEG Framework Output

    • One or more SM files are created per key-frame of every video clip in the database and stored in “.\SMs\” directory created over the root directory of the video clip. Similarly for images, one or more SM files are created image in the database and stored in “.\SMs\” directory created over the root directory of the database file. The naming convention is as follows:
      [indexed video filename]_[KF index]_[SEG module FourCC]_[Sub-Segment Index].pgm
      - for a Key-Frame

      [indexed image filename]_[SEG module FourCC]_[Sub-Segment Index].pgm
      - for an image

    • The sub-segment index is used for enumeration of multiple sub-segmentation masks. For instance if 3 SMs are generated using a specific SEG module (say RSST) for an image; “Image_12”, then the filenames will be:
      Image_12_RSST_1. pgm
      Image_12_RSST_2. pgm

    • Similarly the following three SM filenames are valid for 8th KF of a video clip called “MTV_Clip_12”:
      MTV_Clip_12_KF_8_RSST_0. pgm

      Data Structures in SEG Modules
    • FrameType Enumeration: Enumerates two frame formats to be used between the application and a SEG module. Currently RGB 24 bit and YUV 4:2:0 frame buffers are supported.
    • SEGParam Structure: Created and filled by the SED module for handshaking operation. A MUVIS application (i.e. DbsEditor) calls SEG_Bind function once with a pointer to this structure in run-time. Therefore, SEG module fills the following members of this structure to introduce itself to the application:
      • char seg_name[512]
      • long seg_fourcc
      • unsigned int seg_param_no
      • long* seg_param_fourcc
      • double* seg_param_default
      • FrameType ftype
    • FrameParam Structure: Each time a new frame is fed into SEG module via calling SEG_Extract API function with the frame stored inside the given SegParam structure.
      • unsigned char *buffer
      • int Xs, int Ys

      API Functions in SEG Modules
    • int SEG_Bind(SegParam*): Used for handshaking operation between the MUVIS application (DbsEditor) and the SEG module (DLL). A pointer to SegParam structure is given as a parameter. The SEG module fills the structure to introduce itself to the application. This function should be called only once at the beginning, just after the application links the SEG module in run-time.
    • (double*)SEG_Init: Used to initialize the SEG module. The Segmentation parameters are given in a double array, which has the size defined in the SegParam structure. The SEG module performs necessary initialization operations, i.e. memory allocation, table creation etc. This function should be called once at the beginning after the SEG operation parameters are set in the MUVIS application. This function should return 0 if a problem occurs, and a nonzero value if successfully completed.
    • int Seg_Extract(FrameParam frame, FrameParam SM): Used to extract the SM of a frame (buffer) pointed inside the FrameParam structure. The segmentation parameters are already given before when Seg_Init is called. This function is called to perform segmentation of each and every frame (of an image or video key-frame). Upon completion it fills SM indexes into the buffer of FrameParam structure, which is already created and allocated by the application.
    • int SEG_Exit(SegParam *): Used for both resetting and terminating the SEG module operation. It deallocates the memory spaces for SegParam structure allocated in Seg_Bind function. Also, if Seg_Init has been called already, calling this function with no SegParam pointer (set it to NULL) resets the SEG module to prepare it for further SEG operations. This function should be called at least once while the MUVIS application is closed, but usually it can be called at the end of each SEG operation with different SEG parameters.
Please send questions and bug reports to webm@ster,
Release Date: 01.03.2006
Last Update: 10.09.2013
Copyright 2013 MUVIS