In this project, I worked with a video database that included synchronized audio for each video, each running over ten minutes. The main challenge was to accurately identify and match a 20-40 second video snippet with the original video from the database and determine the starting frame of this snippet within the longer video. Given that the snippet might not be an exact match—owing to variations in size or slight filtering but perceptually similar to a segment in the database—the solution involved creating digital signatures.
First, I preprocessed the videos in the database to generate unique digital signatures for each one. Similarly, I generated a sub-signature for the query video during the demonstration phase. The core task then was to effectively match these sub-signatures with the main signatures, utilizing pattern matching techniques to ensure the query video snippet correctly aligned with its source video in the database.
In this project, I developed video signatures using three techniques:
-
Shot Boundary Details: I identified transitions between shots to capture the structural segmentation of videos, essential for creating unique signatures.
-
Color Analysis: I extracted dominant colors from each frame, aggregating this data to establish a visual theme for each video, crucial for the signature.
-
Sound Analysis: Using wave PCM audio samples, I computed threshold levels and frequencies to quantify audio characteristics, integrating this into the video's overall signature.
These combined techniques enabled accurate matching of video snippets to their source videos based on visual and auditory content.
This project was created by a team of three students for the Multimedia Processing course taught by Parag Havaldar at USC in the Spring of 2024.