Basic problems

• How to store multimedia data?

  Most modern databases (e.g. Postgres) support complex types such as variable-length arrays, images.

  For digital images, BLOBs (binary large objects) define data format (ala C structure), index strategies, operators, and other methods.

• How to specify search?
  • Keyword based

  • (Color) histogram based

  • Edge-feature based
    • Edge pixels
    • Sketches (lines and curves)
    • Shape (junctions, surfaces, polyhedra, ...)

  • Texture based
    • Statistics on gray-level images -> co-occurrence matrix, autocorrelation
    • Edge-based methods -> "edgeness" (i.e., edge density), mean/variance/percentage on edge orientations/separations

  • Motion based (especially in video)

  • Knowledge based
    • geometric models, e.g. lines and corners on soccer fields
    • "Concept queries" in Chabot, e.g. "sunset"

• Possible Improvements:
  • Multiresolution approach (low resolution to be faster, focus on certain areas at higher resolutions for details or motion, ...)

  • Working on compressed image and video. Take advantage of useful information in JPEG, MPEG (DC, AC, motion vectors, ...).