CBR in Video

Video Parsing (Temporal Segmentation)

• Scene breaks/transitions (cuts, fades, dissolves, wipes, etc.)
  • Based on global representation (e.g. colour, intensity histogram)
  • Advantage: insensitive to camera or object motions
  • Disadvantage: miss scene breaks when they have similar distributions

  • Local feature based (e.g. entering/exiting edges in see refernce for Zabih, et al. below)
  • Advantage: capable of using more detailed spatial information
  • Disdvantage: sensitive to moving objects and cameras

• Recover camera motion, zoom, pan

• Key frame extraction often follows the temporal segmentation. In Zhang, et al. (See Refs. below), on average 2-3 key frames are extracted for each shot.

• Salient still - all the frames in a shot are reduced to a single image which is to capture the content and context of the entire shot (difficult and open problem).

Domain Knowledge Based Retrieval

• Example 1: Model-based content extraction of soccer games
  • First, line-feature based models are stored for midfield, left/right goal, corner, etc.

    Then, matching is performed on keyframes for queries such as "a upper-left corner kick".

• Example 2:"Concept queries" in Chabot
  • Choose from menu a query "sunset", the system will generate a query which contains the following:

    
    
    

• Example 3: Automatic indexing of real-time video by NTT, Japan
  • The goal is to generate indexing of 24-hour programs of a TV station:

    Suppose $S_C = \left\{S_1, S_2, ..., S_n\right\}$, where S_i is the time when a scene break occurs, then the set of commercial advertisements can be defined as:

    $\left\{(S_i, S_j) \vert j - i \gt 3, S_j - S_i = 15 (or 30)\right\}$, which states that if in a period of 15 (or 30) seconds there are more than 3 scene breaks, then it is a commercial program.