Video Summarization Using Neural Networks

Video summarization and highlight detection, in particular, is a complex and gradually gaining popularity task among computer vision researchers. The growing interest in this task is justified by the increasing share of video content consumption among Internet users. Therefore, services that aggregate video content need to use not only recommender but also systems that select the most interesting cover and the preview of the video to attract and preserve user’s attention. That problem can be solved with video highlight detection. At the moment, most of the currently existing approaches to this task are based on supervised learning, which requires a lot of effort to get labels for enough videos. In our paper, we implemented an approach based on unsupervised learning, the main idea of which is to use a large number of short and long videos scrapped from Instagram users' posts. We assume that when a user uploads a short video, she tries to show only exciting moments, which cannot be said about a long video because it might contain interesting and uninteresting fragments. We made several domain-specific ranking neural networks and trained them on the mined video files. Also, we presented their comparative results based on the mAP metrics on two public tagged datasets: YouTube Highlights and TVSum.