Temporal and Fine-Grained Pedestrian Action Recognition on Driving Recorder Database - PubMed Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 Feb 20;18(2):627.
doi: 10.3390/s18020627.

Temporal and Fine-Grained Pedestrian Action Recognition on Driving Recorder Database

Hirokatsu Kataoka  1 , Yutaka Satoh  2 , Yoshimitsu Aoki  3 , Shoko Oikawa  4 , Yasuhiro Matsui  5
Affiliations

Temporal and Fine-Grained Pedestrian Action Recognition on Driving Recorder Database

Hirokatsu Kataoka et al. Sensors (Basel). .

Abstract

The paper presents an emerging issue of fine-grained pedestrian action recognition that induces an advanced pre-crush safety to estimate a pedestrian intention in advance. The fine-grained pedestrian actions include visually slight differences (e.g., walking straight and crossing), which are difficult to distinguish from each other. It is believed that the fine-grained action recognition induces a pedestrian intention estimation for a helpful advanced driver-assistance systems (ADAS). The following difficulties have been studied to achieve a fine-grained and accurate pedestrian action recognition: (i) In order to analyze the fine-grained motion of a pedestrian appearance in the vehicle-mounted drive recorder, a method to describe subtle change of motion characteristics occurring in a short time is necessary; (ii) even when the background moves greatly due to the driving of the vehicle, it is necessary to detect changes in subtle motion of the pedestrian; (iii) the collection of large-scale fine-grained actions is very difficult, and therefore a relatively small database should be focused. We find out how to learn an effective recognition model with only a small-scale database. Here, we have thoroughly evaluated several types of configurations to explore an effective approach in fine-grained pedestrian action recognition without a large-scale database. Moreover, two different datasets have been collected in order to raise the issue. Finally, our proposal attained 91.01% on National Traffic Science and Environment Laboratory database (NTSEL) and 53.23% on the near-miss driving recorder database (NDRDB). The paper has improved +8.28% and +6.53% from baseline two-stream fusion convnets.

Keywords: advanced driver-assistance systems (ADAS); driving recorder; fine-grained pedestrian action recognition; two-stream convnets.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflicts of interest.

Figures

Figure 1
Figure 1
Fine-grained pedestrian actions on the self-collected databases: (a) crossing; (b) walking straight; (c) turning; and (d) riding a bicycle. Fine-grained pedestrian action recognition should be an issue in safety systems that have a recognition problem with distinguishing different actions between subtle changes. To improve the recent safety systems such as advanced driver assistance systems (ADAS) and self-driving cars, the concept is very important because a pedestrian intention can be estimated in advance.
Figure 2
Figure 2
Flowchart of our proposed approach: Proposed architecture for fine-grained pedestrian action recognition. We assign two-stream fusion convnets [45] originally proposed by Feichtenhofer et al. The conventional work operates channel-sum with two different convolutional maps in an intermediate layer of spatial- and temporal-stream. After the channel fusion layer (“fusion” in the architecture), we add several convolutional and pooling layers (conv and pool) in order to generate a strong feature, e.g., subtle difference in walking pedestrian. In the classification step, we employ deep convolutional activation features (DeCAF; the first fully-connected layer (FC) with 4096-d vector) to converge the small-scale database by combining with support vector machines (SVM) [46]. Two-stream fusion convnets and DeCAF + SVM are trained with a training-set on self-collected databases.
Figure 3
Figure 3
SVM parameter tuning. (a) relationship between performance rate and SVM parameter on NTSEL; (b) relationship between performance rate and SVM parameter on NDRDB.
Figure 3
Figure 3
SVM parameter tuning. (a) relationship between performance rate and SVM parameter on NTSEL; (b) relationship between performance rate and SVM parameter on NDRDB.
Figure 4
Figure 4
Visual results on NTSEL dataset: the first three lines, there are three success cases as the examples of walking and turnings. The last row shows the failure case in a sequence of a person is riding a bicycle. Especially in the second row, we succeeded with an estimation of pedestrian intention in advance. The turning walking action is important for a safety system.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Geronimo D., Lopez A.M., Sappa A.D., Graf T. Survey of Pedestrian Detection for Advanced Driver Assistance Systems. IEEE Trans. Pattern Anal. Mach. Intell. 2010;32:1239–1258. doi: 10.1109/TPAMI.2009.122. - DOI - PubMed
    1. Benenson R., Omran M., Hosang J., Schiele B. Ten years of pedestrian detection, what have we learned?; Proceedings of the European Conference on Computer Vision Workshop (ECCVW); Zurich, Switzerland. 6–12 September 2014.
    1. Dalal N., Triggs B. Histograms of Oriented Gradients for Human Detection; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR); San Diego, CA, USA. 20–25 June 2005; pp. 886–893.
    1. Viola P., Jones M. Rapid Object Detection using a Boosted Cascaded of Simple Features; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR); Kauai, HI, USA. 8–14 December 2001; pp. 511–518.
    1. Felzenszwalb P., Girshick R., McAllester D., Ramanan D. Object Detection with Discriminatively Trained Part-Based Models. IEEE Trans. Pattern Anal. Mach. Intell. 2010;32:1627–1645. doi: 10.1109/TPAMI.2009.167. - DOI - PubMed

LinkOut - more resources