HHATP: A Lightweight Heterogeneous Hierarchical Attention Model for Trajectory Prediction | SpringerLink
Skip to main content
Springer Nature Link
Log in

HHATP: A Lightweight Heterogeneous Hierarchical Attention Model for Trajectory Prediction

  • Conference paper
  • First Online:
Pattern Recognition and Computer Vision (PRCV 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15040))

Included in the following conference series:

Abstract

Predicting the future trajectories of agents in complex traffic scene is one of the key issues in autonomous driving, requiring reliable and effective predictions for all agents in the scene. Existing trajectory prediction models have achieved high performance on public datasets, but deploying models on vehicles requires both high accuracy and fast computation. It is necessary to balance the complexity of computation and the effectiveness of the structure when designing model. To address the above problem, we proposes a lightweight trajectory prediction model HHATP. Our method is scene-centric and located in the same coordinate system. We use different encoders for the heterogeneous scene objects and the encoded results are then fed into a hierarchical attention module, which considers both global and local interaction to model the relationships between elements. Subsequently, a dynamic weight decoder is used to obtain the trajectories of all agents. Our method achieves good accuracy on the Argoverse dataset and enables fast inference.

This work was supported by the National Key Research and Development Program of China under Grant 2020AAA0108103.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 9380
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 11725
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aydemir, G., Akan, A.K., Güney, F.: Adapt: efficient multi-agent trajectory prediction with adaptation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8295–8305 (2023)

    Google Scholar 

  2. Caesar, H., Bankiti, V., Lang, A.H., Vora, S., Liong, V.E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., Beijbom, O.: Nuscenes: a multimodal dataset for autonomous driving. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11621–11631 (2020)

    Google Scholar 

  3. Chai, Y., Sapp, B., Bansal, M., Anguelov, D.: Multipath: multiple probabilistic anchor trajectory hypotheses for behavior prediction (2019). arXiv:1910.05449

  4. Chang, M.F., Lambert, J., Sangkloy, P., Singh, J., Bak, S., Hartnett, A., Wang, D., Carr, P., Lucey, S., Ramanan, D., et al.: Argoverse: 3d tracking and forecasting with rich maps. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8748–8757 (2019)

    Google Scholar 

  5. Chen, J., Wang, Z., Wang, J., Cai, B.: Q-eanet: Implicit social modeling for trajectory prediction via experience-anchored queries. IET Intell. Transp. Syst. (2023)

    Google Scholar 

  6. Cui, H., Radosavljevic, V., Chou, F.C., Lin, T.H., Nguyen, T., Huang, T.K., Schneider, J., Djuric, N.: Multimodal trajectory predictions for autonomous driving using deep convolutional networks. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 2090–2096. IEEE (2019)

    Google Scholar 

  7. Djuric, N., Radosavljevic, V., Cui, H., Nguyen, T., Chou, F.C., Lin, T.H., Singh, N., Schneider, J.: Uncertainty-aware short-term motion prediction of traffic actors for autonomous driving. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2095–2104 (2020)

    Google Scholar 

  8. Gao, J., Sun, C., Zhao, H., Shen, Y., Anguelov, D., Li, C., Schmid, C.: Vectornet: encoding hd maps and agent dynamics from vectorized representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11525–11533 (2020)

    Google Scholar 

  9. Gilles, T., Sabatini, S., Tsishkou, D., Stanciulescu, B., Moutarde, F.: Thomas: trajectory heatmap output with learned multi-agent sampling (2021). arXiv:2110.06607

  10. Gilles, T., Sabatini, S., Tsishkou, D., Stanciulescu, B., Moutarde, F.: Gohome: Graph-oriented heatmap output for future motion estimation. In: 2022 International Conference on Robotics and Automation (ICRA), pp. 9107–9114. IEEE (2022)

    Google Scholar 

  11. Girgis, R., Golemo, F., Codevilla, F., Weiss, M., D’Souza, J.A., Kahou, S.E., Heide, F., Pal, C.: Latent variable sequential set transformers for joint multi-agent motion prediction (2021). arXiv:2104.00563

  12. Graves, A., Graves, A.: Long short-term memory. In: Supervised Sequence Labelling with Recurrent Neural Networks, pp. 37–45 (2012)

    Google Scholar 

  13. Gu, J., Sun, C., Zhao, H.: Densetnt: end-to-end trajectory prediction from dense goal sets. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15303–15312 (2021)

    Google Scholar 

  14. Jia, X., Wu, P., Chen, L., Liu, Y., Li, H., Yan, J.: Hdgt: heterogeneous driving graph transformer for multi-agent trajectory prediction via scene encoding. IEEE Trans. Pattern Anal. Mach. Intell. (2023)

    Google Scholar 

  15. Liang, M., Yang, B., Hu, R., Chen, Y., Liao, R., Feng, S., Urtasun, R.: Learning lane graph representations for motion forecasting. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16, pp. 541–556. Springer (2020)

    Google Scholar 

  16. Liu, Y., Zhang, J., Fang, L., Jiang, Q., Zhou, B.: Multimodal motion prediction with stacked transformers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7577–7586 (2021)

    Google Scholar 

  17. Mercat, J., Gilles, T., El Zoghby, N., Sandou, G., Beauvois, D., Gil, G.P.: Multi-head attention for multi-modal joint vehicle motion forecasting. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 9638–9644. IEEE (2020)

    Google Scholar 

  18. Ngiam, J., Caine, B., Vasudevan, V., Zhang, Z., Chiang, H.T.L., Ling, J., Roelofs, R., Bewley, A., Liu, C., Venugopal, A., et al.: Scene transformer: a unified architecture for predicting multiple agent trajectories (2021). arXiv:2106.08417

  19. Park, D., Ryu, H., Yang, Y., Cho, J., Kim, J., Yoon, K.J.: Leveraging future relationship reasoning for vehicle trajectory prediction (2023). arXiv:2305.14715

  20. Salzmann, T., Ivanovic, B., Chakravarty, P., Pavone, M.: Trajectron++: Dynamically-feasible trajectory forecasting with heterogeneous data. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVIII 16, pp. 683–700. Springer (2020)

    Google Scholar 

  21. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Adv. Neural Inf. Process. Syst. 30 (2017)

    Google Scholar 

  22. Wang, J., Ye, T., Gu, Z., Chen, J.: Ltp: lane-based trajectory prediction for autonomous driving. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17134–17142 (2022)

    Google Scholar 

  23. Ye, M., Cao, T., Chen, Q.: Tpcn: temporal point cloud networks for motion forecasting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11318–11327 (2021)

    Google Scholar 

  24. Yu, C., Ma, X., Ren, J., Zhao, H., Yi, S.: Spatio-temporal graph transformer networks for pedestrian trajectory prediction. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XII 16, pp. 507–523. Springer (2020)

    Google Scholar 

  25. Yuan, Y., Weng, X., Ou, Y., Kitani, K.M.: Agentformer: agent-aware transformers for socio-temporal multi-agent forecasting. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9813–9823 (2021)

    Google Scholar 

  26. Zhao, H., Gao, J., Lan, T., Sun, C., Sapp, B., Varadarajan, B., Shen, Y., Shen, Y., Chai, Y., Schmid, C., et al.: Tnt: target-driven trajectory prediction. In: Conference on Robot Learning, pp. 895–904. PMLR (2021)

    Google Scholar 

  27. Zhou, Z., Wang, J., Li, Y.H., Huang, Y.K.: Query-centric trajectory prediction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17863–17873 (2023)

    Google Scholar 

  28. Zhou, Z., Ye, L., Wang, J., Wu, K., Lu, K.: Hivt: hierarchical vector transformer for multi-agent motion prediction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8823–8833 (2022)

    Google Scholar 

  29. Zhu, L., Wang, X., Ke, Z., Zhang, W., Lau, R.W.: Biformer: vision transformer with bi-level routing attention. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10323–10333 (2023)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei, 230031, China

    Zeyu Lai, Xingliang Zhu, Chunmeng Yang & Bin Kong

  2. University of Science and Technology of China, Hefei, 230026, China

    Zeyu Lai, Xingliang Zhu & Chunmeng Yang

  3. Anhui Engineering Laboratory for Intelligent Driving Technology and Application, Hefei, 230031, China

    Bin Kong

Authors
  1. Zeyu Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xingliang Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chunmeng Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bin Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bin Kong .

Editor information

Editors and Affiliations

  1. Peking University, Beijing, China

    Zhouchen Lin

  2. Nankai University, Tianjin, China

    Ming-Ming Cheng

  3. Chinese Academy of Sciences, Beijing, China

    Ran He

  4. Xinjiang University, Ürümqi, Xinjiang, China

    Kurban Ubul

  5. Xinjiang University, Ürümqi, China

    Wushouer Silamu

  6. Peking University, Beijing, China

    Hongbin Zha

  7. Tsinghua University, Beijing, China

    Jie Zhou

  8. Chinese Academy of Sciences, Beijing, China

    Cheng-Lin Liu

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lai, Z., Zhu, X., Yang, C., Kong, B. (2025). HHATP: A Lightweight Heterogeneous Hierarchical Attention Model for Trajectory Prediction. In: Lin, Z., et al. Pattern Recognition and Computer Vision. PRCV 2024. Lecture Notes in Computer Science, vol 15040. Springer, Singapore. https://doi.org/10.1007/978-981-97-8792-0_9

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-8792-0_9

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-8791-3

  • Online ISBN: 978-981-97-8792-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics