HackTheNorth
2021
Thomas J. Hu Bhavini Rathod
Nick Pysklywec Andy Huang
Table of Contents
01
02
03
04
Challenge
Solution
Demo
Future Plans
Challenge
01
CANSOFCOM - Video Summarization
To develop automated video summarization tools that can detect objects and events of interest in a video and make this content searchable. The participants must develop an application that can ingest a long video (e.g. one hour of a person walking in a city) and produce a searchable summary of this video.
Solution
02
Technology We Used
Pipeline
Colab
YouTube API
=> Image Category
=> Scrape google images
=> Transfer Learn
Detectron2 based transfer learning on special ops relevant categories
Simulation of searching & query process.
React
Frontend.
Displays a search bar, videos, and video summary.
Markup We Proposed
Step 1
Step 3
Step 2
03
Use cases
Improvements
04
Improvements
Analyze Longer Videos
Smart sampling, where interesting moments are sampled at higher rates
Scalable & Performant
noSQL DB indexed on Salient keywords�cold & hot storage
Video Chapters
Highlighting the location of the searched object and the amount of time it appears in the video
Activity Recognition
Patrolling, digging, loading & unloading.
Direction of movement.
GPS
01
03
04
02
Thanks!
Bhavini Rathod
Andy Huang
Thomas J. Hu
Nick Pysklywec