{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,1,22]],"date-time":"2024-01-22T23:10:03Z","timestamp":1705965003000},"reference-count":40,"publisher":"Wiley","issue":"2","license":[{"start":{"date-parts":[[2012,11,20]],"date-time":"2012-11-20T00:00:00Z","timestamp":1353369600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Int J Network Mgmt"],"published-print":{"date-parts":[[2013,3]]},"abstract":"SUMMARY<\/jats:title>Today's network control systems have very limited ability to adapt to changing network conditions. The addition of reinforcement learning\u2010based network management agents can improve quality of service by reconfiguring the network layer protocol parameters in response to observed network performance conditions. This paper presents a closed\u2010loop approach to tuning the layer three protocol based upon current and previous network state observations, specifically the Hello Interval and Active Route Timeout parameters of the AODV routing protocol (AODV\u2010Q). Simulation results demonstrate that the self\u2010configuration method proposed here demonstrably improves the performance of the original Ad\u2010Hoc On\u2010Demand Distance Vector (AODV) protocol, reducing protocol overhead by 43% and end\u2010to\u2010end delay 29% while increasing the packet delivery ratio by up to 11%. Copyright \u00a9 2012 John Wiley & Sons, Ltd.<\/jats:p>","DOI":"10.1002\/nem.1819","type":"journal-article","created":{"date-parts":[[2012,11,20]],"date-time":"2012-11-20T13:44:10Z","timestamp":1353419050000},"page":"119-136","source":"Crossref","is-referenced-by-count":6,"title":["Self\u2010adapting protocol tuning for multi\u2010hop wireless networks using Q\u2010learning"],"prefix":"10.1002","volume":"23","author":[{"given":"Dan","family":"Marconett","sequence":"first","affiliation":[{"name":"Department of Computer Science University of California, Davis Davis CA 95616 USA"}]},{"given":"Minsoo","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering University of California Davis Davis CA 95616 USA"}]},{"given":"Xiaohui","family":"Ye","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering University of California Davis Davis CA 95616 USA"}]},{"given":"Rao","family":"Vemuri","sequence":"additional","affiliation":[{"name":"Department of Computer Science University of California, Davis Davis CA 95616 USA"}]},{"given":"S. J. B.","family":"Yoo","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering University of California Davis Davis CA 95616 USA"}]}],"member":"311","published-online":{"date-parts":[[2012,11,20]]},"reference":[{"key":"e_1_2_8_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSAC.2010.100406"},{"key":"e_1_2_8_3_1","doi-asserted-by":"crossref","unstructured":"LauferR SalonidisT LundgrenH Le GuyadecP.XPRESS: a cross\u2010layer backpressure architecture for wireless multi\u2010hop networks. InProceedings of ACM International Conference on Mobile Computing and Networking(MobiCom) 2011.","DOI":"10.1145\/2030613.2030620"},{"key":"e_1_2_8_4_1","doi-asserted-by":"crossref","unstructured":"WangP WangT.Adaptive routing for sensor networks using reinforcement learning. InProceedings of the Sixth IEEE International Conference on Computer and Information Technology(CIT'06) 2006.","DOI":"10.1109\/CIT.2006.34"},{"key":"e_1_2_8_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2006.273099"},{"key":"e_1_2_8_6_1","first-page":"279","volume-title":"Machine Learning","author":"Watkins C","year":"1992"},{"key":"e_1_2_8_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/98.760423"},{"key":"e_1_2_8_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCA.2005.846390"},{"key":"e_1_2_8_9_1","doi-asserted-by":"crossref","unstructured":"BasharA ParrG McCleanS ScotneyB NauckD.Machine learning based call admission control approaches: a comparative study. InProceedings of 6th IEEE\/IFIP International Conference on Network and Service Management October2010.","DOI":"10.1109\/CNSM.2010.5691261"},{"key":"e_1_2_8_10_1","doi-asserted-by":"crossref","unstructured":"BrandC WolhuterR.Traffic class prediction and prioritization on a diversified IP network using machine learning. InProceedings of IEEE Global Communications Conference(Globecom'09) Workshops November2009.","DOI":"10.1109\/GLOCOMW.2009.5360759"},{"key":"e_1_2_8_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2004.1266295"},{"key":"e_1_2_8_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2005.1561929"},{"key":"e_1_2_8_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2006.1668385"},{"key":"e_1_2_8_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2005.1561928"},{"key":"e_1_2_8_15_1","unstructured":"YinL UttamchandaniS PalmerJ KatzR AghaG.AutoLoop: automated action selection in the \u2018observe\u2013analyze\u2013act\u2019 loop for storage systems. InProceedings of Sixth IEEE International Workshop on Policies for Distributed Systems and Networks June2005;129\u2013138."},{"key":"e_1_2_8_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/COM-M.2006.248156"},{"key":"e_1_2_8_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/MWC.2006.1632476"},{"key":"e_1_2_8_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2006.1632660"},{"key":"e_1_2_8_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/MNET.2008.4435899"},{"key":"e_1_2_8_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNET.2007.912064"},{"key":"e_1_2_8_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2008.4689245"},{"key":"e_1_2_8_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2009.4785392"},{"key":"e_1_2_8_23_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.comnet.2004.12.001"},{"key":"e_1_2_8_24_1","doi-asserted-by":"crossref","unstructured":"ClausenT JacquetP.Optimized Link State Routing Protocol (OLSR).IETF RFC 3626 2003.","DOI":"10.17487\/rfc3626"},{"key":"e_1_2_8_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-585-29603-6_5"},{"key":"e_1_2_8_26_1","doi-asserted-by":"crossref","unstructured":"PerkinsC Belding\u2010RoyerE DasS.Ad hoc on\u2010demand distance vector (AODV) routing.RFC 3561 IETF MANET Working Group August2003.","DOI":"10.17487\/rfc3561"},{"key":"e_1_2_8_27_1","unstructured":"HaasZJ PearlmanM SamarP.The Zone Routing Protocol (ZRP) for ad hoc networks. IETF Internet Draft 2002."},{"key":"e_1_2_8_28_1","doi-asserted-by":"publisher","DOI":"10.1002\/nem.463"},{"key":"e_1_2_8_29_1","doi-asserted-by":"crossref","unstructured":"KleinA.Performance comparison and evaluation of AODV OLSR and SBR in mobile ad\u2010hoc networks. InProceedings of 3rd International Symposium on Wireless Pervasive Computing(ISWPC 2008) May2008;571\u2013575.","DOI":"10.1109\/ISWPC.2008.4556273"},{"key":"e_1_2_8_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSAC.2004.829351"},{"key":"e_1_2_8_31_1","doi-asserted-by":"crossref","unstructured":"VaddeKK SyrotiukVR.On timers of routing protocols in Manets. InProceedings of Third International Conference on Ad Hoc Networks and Wireless(AdHoc Now'04) July2004;330\u2013335.","DOI":"10.1007\/978-3-540-28634-9_29"},{"key":"e_1_2_8_32_1","unstructured":"TanHX SeahWKG.Dynamically adapting mobile ad hoc routing protocols to improve scalability. InProceedings of the IASTED International Conference on Communication Systems and Networks(CSN2004) September2004."},{"key":"e_1_2_8_33_1","unstructured":"LiW HanJ.Dynamic wireless sensor network parameters optimization adapting different node mobility. InProceedings of IEEE Aerospace Conference March2010;1\u20137."},{"key":"e_1_2_8_34_1","unstructured":"XingT LiuY TangB WuF.Dynamic\u2010adjusting AODV routing protocol based on max hop count. InProceedings of IEEE International Conference on Wireless Communications Networking and Information Security(WCNIS) June2010;535\u2013539."},{"key":"e_1_2_8_35_1","doi-asserted-by":"crossref","unstructured":"LeeM YeX MarconettD JohnsonS VemuriR YooSJB.Autonomous network management using cooperativ learning for network\u2010wide load balancing in heterogeneous networks. InProceedings of IEEE Global Telecommunications Conference December2008;1\u20135.","DOI":"10.1109\/GLOCOM.2008.ECP.489"},{"key":"e_1_2_8_36_1","doi-asserted-by":"crossref","unstructured":"LeeM MarconettD YeX YooSJB.Cognitive network management with reinforcement learning for wireless mesh networks. InProceedings of the 7th IEEE International Conference on IP Operations and Management 2007;168\u2013179.","DOI":"10.1007\/978-3-540-75853-2_15"},{"key":"e_1_2_8_37_1","unstructured":"WangH CuiL.An enhanced AODV for mobile ad hoc network. InProceedings of International Conference on Machine Learning and Cybernetics Vol. 2 July2008;1135\u20131140."},{"key":"e_1_2_8_38_1","unstructured":"RichardC PerkinsC WestphalC.Defining an optimal active route timeout for the AODV routing protocol. InProceedings of Sensor and Ad Hoc Communications and Networks(IEEE SECON) 2005."},{"key":"e_1_2_8_39_1","unstructured":"BoyanJ LittmanM.Packet routing in dynamically changing networks: a reinforcement learning approach. InAdvances In Neural Information Processing Systems 1994."},{"key":"e_1_2_8_40_1","unstructured":"NS\u20102 Network Simulator (August 2012). Available:http:\/\/nsnam.isi.edu\/nsnam\/index.php\/User_Information[3 November2012]."},{"key":"e_1_2_8_41_1","unstructured":"DeardenR FriedmanN AndreD.Model based Bayesian exploration. InProceedings of Conference on Uncertainty in Artificial Intelligence(UAI'99) 1999;150\u2013159."}],"container-title":["International Journal of Network Management"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fnem.1819","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/nem.1819","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,8]],"date-time":"2023-10-08T20:27:54Z","timestamp":1696796874000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/nem.1819"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,11,20]]},"references-count":40,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2013,3]]}},"alternative-id":["10.1002\/nem.1819"],"URL":"https:\/\/doi.org\/10.1002\/nem.1819","archive":["Portico"],"relation":{},"ISSN":["1055-7148","1099-1190"],"issn-type":[{"value":"1055-7148","type":"print"},{"value":"1099-1190","type":"electronic"}],"subject":[],"published":{"date-parts":[[2012,11,20]]}}}