A task allocation algorithm based on reinforcement learning in spatio-temporal crowdsourcing | Applied Intelligence Skip to main content

Advertisement

Springer Nature Link
Log in

A task allocation algorithm based on reinforcement learning in spatio-temporal crowdsourcing

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

With the pervasiveness of dynamic task allocation in sharing economy applications, online bipartite graph matching has attracted more and more research attention. In sharing economy applications, crowdsourcing platforms need to allocate tasks to workers dynamically. Previous studies have low allocation utility. To increase the allocation utility of the Spatio-temporal crowdsourcing system, this paper proposes a dynamic delay bipartite matching(DDBM) problem, and designs Value Based Task Allocation(VBTA) and Policy Gradient Based Task Allocation(PGTA) frameworks respectively. According to the current state, VBTA and PGTA could enhance the allocation utility by selecting appropriate thresholds. The convergence of the algorithm is proved. Extensive experimental results on two real datasets demonstrate that the proposed algorithms are superior to the existing algorithms in effectiveness and efficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Algorithm 1
Fig. 7
Algorithm 2
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Ganti RK, Ye F, Lei H (2011) Mobile crowdsensing: current state and future challenges. IEEE Commun Mag 49(11):32–39

    Article  Google Scholar 

  2. Bin G, Zhu W, Zhiwen Y, Yu W, Neil Y (2015) Mobile crowd sensing and computing: The review of an emerging human-powered sensing paradigm. ACM Comput Surv (CSUR), 48(1)

  3. Amazon Mechanical Turks. https://www.mturk.com/

  4. Upwork. https://www.upwork.com/

  5. Tong Y, Zhou Z, Zeng Y, Chen L, Shahabi C (2020) Spatial crowdsourcing: a survey. VLDB J, 29(1):217–250

    Article  Google Scholar 

  6. Shi D, Tong Y, Zhou Z, Song B, Lv W, Yang Q (2021) Learning to assign: towards fair task assignment in large-scale ride hailing. In: Zhu F, Ooi BC, Miao C (eds) KDD ’21: The 27th ACM SIGKDD conference on knowledge discovery and data mining, virtual event, Singapore, August 14-18, 2021. ACM, pp 3549–3557

  7. Reddy S, Shilton K, Burke J, Estrin D, Srivastava MB (2009) Using context annotated mobility profiles to recruit data collectors in participatory sensing. In: Location and context awareness, 4th international symposium, LoCA 2009, Tokyo, Japan, May 7-8, Proceedings

  8. Reddy S, Estrin D, Srivastava M Recruitment framework for participatory sensing data collections. In: Pervasive computing, international conference. Pervasive, Helsinki, Finland, May

  9. Cardone G, Foschini L, Bellavista P, Corradi A, Borcea C (2013) Fostering participaction in smart cities: a geo-social crowdsensing platform. IEEE Communications Magazine 51(6):N

  10. Zhang M, Yang P, Tian C, Tang S, Gao X, Wang B, Xiao F (2016) Quality-aware sensing coverage in budget-constrained mobile crowdsensing networks. IEEE Trans Veh Technol 65(9):7698–7707

    Article  Google Scholar 

  11. Guo B, Liu Y, Wenle W, Zhiwen Y, Han Q (2017) Activecrowd: a framework for optimized multi-task allocation in mobile crowdsensing systems. IEEE Trans Human-Mach Syst PP(3):392–403

    Article  Google Scholar 

  12. Kazemi L, Shahabi C Geocrowd: enabling query answering with spatial crowdsourcing. In: Proceedings of the 20th international conference on advances in geographic information systems

  13. Cheng P, Lian X, Chen L, Shahabi C (2014) Prediction-based task assignment in spatial crowdsourcing. In: 2017 IEEE 33rd International Conference on Data Engineering (ICDE). https://doi.org/10.1109/ICDE.2017.146, pp 997–1008

  14. Peng C, Lian X, Chen Z, Chen L, Zhao J (2014) Reliable diversity-based spatial crowdsourcing by moving workers. Proceedings of the VLDB Endowment

  15. Wang Y, Tong Y, Long C, Xu P, Xu K, Lv W Adaptive dynamic bipartite graph matching: a reinforcement learning approach. In: 2019 IEEE 35th International conference on data engineering (ICDE)

  16. Tianshu S, Xu K, Li J, Li Y, Tong Y (2019) Multi-skill aware task assignment in real-time spatial crowdsourcing. GeoInformatica

  17. Liu JX, Xu K (2018) Budget-aware online task assignment in spatial crowdsourcing

  18. Bhaskar N, Mohan Kumar P, Arokia Renjit J (2020) Evolutionary fuzzy-based gravitational search algorithm for query optimization in crowdsourcing system to minimize cost and latency. Computational Intelligence

  19. Nam S, Lee M, Heo C, Choi KS (2020) Cost-effective multi-task crowdsourcing method for knowledge extraction. KIISE Trans Comput Pract 26(11):507–512

    Article  Google Scholar 

  20. Chen Z, Cheng P, Chen L, Lin X, Shahabi C (2020) Fair task assignment in spatial crowdsourcing. Proc VLDB Endow 13(11):2479–2492

    Article  Google Scholar 

  21. Wang Y, Gao Y, Li Y, Tong X (2020) A worker-selection incentive mechanism for optimizing platform-centric mobile crowdsourcing systems. Comput Netw 171:107144

    Article  Google Scholar 

  22. Wang Y, Cai Z, Zhan Z-H, Zhao B, Tong X, Qi L (2020) Walrasian equilibrium-based multiobjective optimization for task allocation in mobile crowdsourcing. IEEE Trans Comput Soc Syst 7(4):1033–1046

    Article  Google Scholar 

  23. Puterman ML (1994) Markov decision processes: Discrete stochastic dynamic programming. Wiley

  24. Sutton RS, Barto AG (2018) Reinforcement earning: an introduction. MIT Press

  25. Bhatti SS, Gao X, Chen G (2020) General framework, opportunities and challenges for crowdsourcing techniques: a comprehensive survey. J Syst Softw 167:110611

    Article  Google Scholar 

  26. Gong W, Zhang B, Li C (2018) Task assignment in mobile crowdsensing: present and future directions. IEEE Netw 32(4):100–107

    Article  Google Scholar 

  27. Tong Y, Zhou Z (2018) Dynamic task assignment in spatial crowdsourcing. SIGSPATIAL Special

  28. Tong Y, She J, Ding B, Wang L, Chen L (2016) Online mobile micro-task allocation in spatial crowdsourcing. In: 2016 IEEE 32nd international conference on data engineering (ICDE), pp 49–60

  29. Tong Y, Wang L, Zhou Z, Ding B, Chen L, Ye J, Ke X u (2017) Flexible online task assignment in real-time spatial data. Proc Vldb Endow 10(11):1334–1345

    Article  Google Scholar 

  30. Sun Z, Wang Y, Cai Z, Liu T, Jiang N (2021) A two tage privacy protection mechanism based on blockchain in mobile crowdsourcing. International Journal of Intelligent Systems

  31. Liu T, Wang Y, Li Y, Tong X, Qi L, Jiang N (2020) Privacy protection based on stream cipher for spatiotemporal data in iot. IEEE Internet Things J 7(9):7928–7940

    Article  Google Scholar 

  32. Wang L, Yang D, Han X, Zhang D, Ma X (2021) Mobile crowdsourcing task allocation differential-and-distortion geo-obfuscation. IEEE Trans Dependable Secur Comput 18(2):967–981

    Article  Google Scholar 

  33. To H, Shahabi C, Xiong L (2018) Privacy-preserving online task assignment in spatial crowdsourcing with untrusted server. In: 2018 IEEE 34th International conference on data engineering (ICDE), pp 833–844

  34. Cai Z, He Z, Guan X, Li Y (2018) Collective data-sanitization for preventing sensitive information inference attacks in social networks. IEEE Transactions on Dependable & Secure Computing, 1–1

  35. Cai Z, Zheng X (2018) A private and efficient mechanism for data uploading in smart cyber-physical systems. IEEE Transactions on Network Science & Engineering, 1–1

  36. Lin H, Garg S, Hu J, Kaddoum G, Peng M, Hossain MS (2020) Blockchain and deep reinforcement learning empowered spatial crowdsourcing in software-defined internet of vehicles. IEEE Trans Intell Transp Syst, 1–10

  37. Wang Y, Tong Y, Long C, Xu P, Xu K, Lv W Adaptive dynamic bipartite graph matching: A reinforcement learning approach. In: 2019 IEEE 35th international conference on data engineering (ICDE)

  38. Liu W, Wang L, En W, Yang Y, Zeghlache D, Zhang D (2019) Reinforcement learning-based cell selection in sparse mobile crowdsensing. Comput Netw 161:102–114

    Article  Google Scholar 

  39. Xi T, Song W (2020) Task allocation for mobile crowdsensing with deep reinforcement learning. In: 2020 IEEE wireless communications and networking conference, WCNC 2020, Seoul, Korea (South), May 25-28, 2020, pp 1–7. IEEE

  40. Yunfan H u, Wang J, Bo W u, Helal S (2020) Participants selection for from-scratch mobile crowdsensing via reinforcement learning. In: IEEE International conference on pervasive computing and communications (PerCom)

  41. Liu CH, Dai Z, Zhao Y, Crowcroft J, Wu D, Leung KK (2021) Distributed and energy-efficient mobile crowdsensing with charging stations by deep reinforcement learning. IEEE Trans M Comput 20 (1):130–146

    Article  Google Scholar 

  42. Liu CH, Zhao Y, Dai Z, Yuan Y, Wang G, Wu D, Leung KK (2020) Curiosity-driven energy-efficient worker scheduling in vehicular crowdsourcing: a deep reinforcement learning approach. In: 2020 IEEE 36th International conference on data engineering (ICDE), pp 25–36

  43. Song T, Tong Y, Wang L, She J, Yao B, Chen L, Ke X u (2017) Trichromatic online matching in real-time spatial crowdsourcing. In: 2017 IEEE 33rd international conference on data engineering (ICDE), pp 1009–1020. IEEE

  44. Google Map Api. https://developers.google.com/maps/

  45. Watkins C, Dayan P Q-learning. Machine learning

  46. Yelp. https://www.yelp.com/dataset

  47. gMission. http://gmission.github.io/

  48. Tong Y, Zeng Y, Ding B, Wang L, Chen L (2021) Two-sided online micro-task assignment in spatial crowdsourcing. IEEE Trans Knowl Data Eng 33(5):2295–2309

    Google Scholar 

Download references

Acknowledgments

This work was supported by National Natural Science Foundation of China under Grants No.61472095.

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Harbin Engineering University, 145 Nantong Street, Harbin, 150000, Heilongjiang, China

    Bingxu Zhao, Hongbin Dong & Tingwei Pan

  2. School of Computer and Control Engineering, Yantai University, 30 Qingquan Road, Yantai, 264000, Shandong, China

    Yingjie Wang

Authors
  1. Bingxu Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongbin Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yingjie Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tingwei Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongbin Dong.

Ethics declarations

Ethics approval

This article does not contain any studies with human participants or animal performed by any of the authors.

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, B., Dong, H., Wang, Y. et al. A task allocation algorithm based on reinforcement learning in spatio-temporal crowdsourcing. Appl Intell 53, 13452–13469 (2023). https://doi.org/10.1007/s10489-022-04151-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04151-6

Keywords

Search

Navigation