Improved action proposals using fine-grained proposal features with recurrent attention models

Recent models for the temporal action proposal task show that local properties can be an alternative to the region proposal network (RPN) for generating good proposal candidates on untrimmed videos. In this study, we devise an RPN model with a new two-stage pipeline and a new joint scoring function for temporal proposals. The evaluation of local properties is integrated into our RPN model to search for the best proposal candidates that can be distinguished mainly in fine details of proposal regions. Our network models proposals in multiple scales using two recurrent neural network layers with attention mechanisms. We observe that joint training of the RPN with local clues and multi-scale modeling of proposals with recurrent attention mechanisms improve the performance of the proposal generation task. Our model yields state-of-the-art results on the THUMOS-14 and comparable results on the ActivityNet-1.3 datasets.