{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,31]],"date-time":"2024-08-31T06:30:09Z","timestamp":1725085809756},"reference-count":52,"publisher":"Wiley","issue":"3","license":[{"start":{"date-parts":[[2021,6,19]],"date-time":"2021-06-19T00:00:00Z","timestamp":1624060800000},"content-version":"am","delay-in-days":365,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#am"},{"start":{"date-parts":[[2020,6,19]],"date-time":"2020-06-19T00:00:00Z","timestamp":1592524800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"funder":[{"DOI":"10.13039\/501100002322","name":"Coordena\u00e7\u00e3o de Aperfei\u00e7oamento de Pessoal de N\u00edvel Superior","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100002322","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100011875","name":"Minist\u00e9rio da Ci\u00eancia, Tecnologia, Inova\u00e7\u00f5es e Comunica\u00e7\u00f5es","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100011875","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003593","name":"Conselho Nacional de Desenvolvimento Cient\u00edfico e Tecnol\u00f3gico","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100003593","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100017580","name":"Rede Nacional de Ensino e Pesquisa","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100017580","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100005230","name":"Organization of American States","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100005230","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001807","name":"Funda\u00e7\u00e3o de Amparo \u00e0 Pesquisa do Estado de S\u00e3o Paulo","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100001807","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Int J Network Mgmt"],"published-print":{"date-parts":[[2021,5]]},"abstract":"Summary<\/jats:title>Since the term Internet of Things (IoT) was coined by Kevin Ashton in 1999, a number of middleware platforms have been developed to cope with important challenges such as the integration of different technologies. In this context of heterogeneous technologies, IoT message brokers become critical elements for the proper function of smart systems and wireless sensor networks (WSN) infrastructures. There are several evaluations made on IoT messaging middleware performance. Nevertheless, most of them ignore crucial aspects of the IoT context that also need to be included, such as reliability and other qualitative aspects. Thus, in this article, we propose a methodology for classification and evaluation of IoT brokers to help the scientific community and technology industry on evaluating them according to their interests, without leaving out important aspects for the context of smart environments. Our methodology bases its qualitative evaluations on the ISO\/IEC 25000 (SQuaRE) set of standards and its quantitative evaluations on Jain's process for performance evaluation. We developed a case study to illustrate our proposal with 12 different open\u2010source brokers, validating the feasibility of our methodological approach.<\/jats:p>","DOI":"10.1002\/nem.2115","type":"journal-article","created":{"date-parts":[[2020,6,19]],"date-time":"2020-06-19T10:32:00Z","timestamp":1592562720000},"update-policy":"http:\/\/dx.doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Classification and evaluation of IoT brokers: A methodology"],"prefix":"10.1002","volume":"31","author":[{"ORCID":"http:\/\/orcid.org\/0000-0002-8750-3406","authenticated-orcid":false,"given":"Eddas","family":"Bertrand\u2010Martinez","sequence":"first","affiliation":[{"name":"GERCOM \u2010 Research Group on Networks and Multimedia Communications Federal University of Par\u00e1 Bel\u00e9m Brazil"}]},{"given":"Phelipe","family":"Dias Feio","sequence":"additional","affiliation":[{"name":"GERCOM \u2010 Research Group on Networks and Multimedia Communications Federal University of Par\u00e1 Bel\u00e9m Brazil"}]},{"given":"Vagner de","family":"Brito Nascimento","sequence":"additional","affiliation":[{"name":"GERCOM \u2010 Research Group on Networks and Multimedia Communications Federal University of Par\u00e1 Bel\u00e9m Brazil"}]},{"given":"Fabio","family":"Kon","sequence":"additional","affiliation":[{"name":"Department of Computer Science University of S\u00e3o Paulo S\u00e3o Paulo Brazil"}]},{"given":"Ant\u00f4nio","family":"Abel\u00e9m","sequence":"additional","affiliation":[{"name":"GERCOM \u2010 Research Group on Networks and Multimedia Communications Federal University of Par\u00e1 Bel\u00e9m Brazil"}]}],"member":"311","published-online":{"date-parts":[[2020,6,19]]},"reference":[{"key":"e_1_2_8_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2014.2337336"},{"key":"e_1_2_8_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.tele.2018.04.004"},{"key":"e_1_2_8_4_1","unstructured":"EllsmoorJ.Smart Cities: The Future Of Urban Development.https:\/\/www.forbes.com\/sites\/jamesellsmoor\/2019\/05\/19\/smart-cities-the-future-of-urban-development\/\\#7684ffa12f90;2019."},{"key":"e_1_2_8_5_1","doi-asserted-by":"publisher","DOI":"10.1080\/10630732.2011.601117"},{"key":"e_1_2_8_6_1","doi-asserted-by":"crossref","unstructured":"Del EsposteAM KonF CostaFM LagoN.InterSCity: a scalable microservice\u2010based open source platform for smart cities. In Proceedings of the 6th International Conference on Smart Cities and Green ICT Systems (SMARTGREENS 2017). SCITEPRESS \u2010 Science and Technology Publications Lda Setubal PRT 2017:35\u201046.https:\/\/doi.org\/10.5220\/0006306200350046","DOI":"10.5220\/0006306200350046"},{"key":"e_1_2_8_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2016.10.031"},{"key":"e_1_2_8_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3286719.3286728"},{"key":"e_1_2_8_9_1","unstructured":"Unicamp.Smart Campus \u2010 Unicamp.https:\/\/smartcampus.prefeitura.unicamp.br\/(Portuguese)."},{"key":"e_1_2_8_10_1","first-page":"1","volume-title":"Choice of effective messaging protocols for IoT systems: MQTT, CoAP, AMQP and HTTP","author":"Naik N","year":"2017"},{"key":"e_1_2_8_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3292674"},{"key":"e_1_2_8_12_1","doi-asserted-by":"publisher","DOI":"10.1049\/cp.2015.0714"},{"key":"e_1_2_8_13_1","unstructured":"ISO\/IEC.Systems and software engineering \u2013 systems and software quality requirements and evaluation (SQuaRE) \u2013 guide to SQuaRE. International Standard ISO\/IEC 25000:2014;2014."},{"key":"e_1_2_8_14_1","volume-title":"The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation and Modeling","author":"Raj J","year":"1991"},{"key":"e_1_2_8_15_1","doi-asserted-by":"publisher","DOI":"10.1155\/2017\/9324035"},{"key":"e_1_2_8_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2015.2498900"},{"key":"e_1_2_8_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICITEED.2017.8250510"},{"key":"e_1_2_8_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3242102.3243317"},{"key":"e_1_2_8_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/LANMAN.2017.7972165"},{"key":"e_1_2_8_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/IOT.2010.5678443"},{"key":"e_1_2_8_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/FiCloud.2014.27"},{"key":"e_1_2_8_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/NGMAST.2014.66"},{"key":"e_1_2_8_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/CloudNet.2017.8071523"},{"key":"e_1_2_8_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/INCIT.2017.8257870"},{"key":"e_1_2_8_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3017995.3017997"},{"key":"e_1_2_8_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-95171-3_47"},{"key":"e_1_2_8_27_1","unstructured":"Scalagent.Benchmark of MQTT servers;2015."},{"key":"e_1_2_8_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/GCIoT.2018.8620130"},{"key":"e_1_2_8_29_1","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1007\/s12243-016-0537-4","article-title":"Meeting IoT platform requirements with open pub\/sub solutions","volume":"72","author":"Daniel H","year":"2017","journal-title":"Annal Tel"},{"key":"e_1_2_8_30_1","doi-asserted-by":"publisher","DOI":"10.1007\/s40860-018-0056-3"},{"key":"e_1_2_8_31_1","unstructured":"European Comission.The Lighthouse Project IoT\u2010A.https:\/\/cordis.europa.eu\/project\/id\/257521 FP7\u2010ICT \u2010 Specific Programme \u201cCooperation\u201d: Information and communication technologies;2013."},{"key":"e_1_2_8_32_1","unstructured":"GlinzM.Software Product Quality.https:\/\/www.ifi.uzh.ch\/dam\/jcr:8e9cd2bb-60de-4ce7-b17e-8f994744d4cd\/06_product_q.pd;2014."},{"key":"e_1_2_8_33_1","first-page":"592","volume-title":"In Proceedings of the 2nd international conference on Software engineering (ICSE'76)","author":"Boehm BW","year":"1976"},{"key":"e_1_2_8_34_1","unstructured":"BrunstH MuellerM.Performance Analysis of Computer Systems: Requirements metrics techniques and mistakes.https:\/\/tu-dresden.de\/zih\/ressourcen\/dateien\/lehre\/ws1112\/lars\/vorlesungen\/lars_lecture_02_requirements-metrics-techniques.pdf.2019."},{"key":"e_1_2_8_35_1","unstructured":"ISO\/IEC.Systems and software engineering \u2013 systems and software quality requirements and evaluation (SQuaRE) \u2013 system and software quality models. International Standard ISO\/IEC 25010:2011;2011."},{"key":"e_1_2_8_36_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2013.01.083"},{"key":"e_1_2_8_37_1","unstructured":"McCallJ MatsumotoM.Software Quality Measurement Manual. Rome Air Development Center RADC\u2010TR\u201080\u2010109\u2010Vol\u20102;1980."},{"key":"e_1_2_8_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/1985793.1985977"},{"key":"e_1_2_8_39_1","unstructured":"ISO\/IEC.Systems and software engineering \u2010 systems and software quality requirements and evaluation (SQuaRE) \u2010 evaluation process. International Standard ISO\/IEC 25040:2011;2011."},{"key":"e_1_2_8_40_1","unstructured":"Eclipse Foundation.Mosquitto \u2010 An open source MQTT broker.https:\/\/mosquitto.org\/"},{"key":"e_1_2_8_41_1","unstructured":"Apache Foundation.ActiveMQ \u2010 Flexible & Powerful Open Source Multi\u2010Protocol Messaging.https:\/\/activemq.apache.org\/"},{"key":"e_1_2_8_42_1","unstructured":"Apache Foundation.The Apollo Project.https:\/\/github.com\/apache\/activemq-apollo"},{"key":"e_1_2_8_43_1","unstructured":"Pivotal Software.RabbitMQ \u2010 Messaging that just works.https:\/\/www.rabbitmq.com\/"},{"key":"e_1_2_8_44_1","unstructured":"Eclipse Foundation.Moquette MQTT Broker.https:\/\/moquette-io.github.io\/moquette\/"},{"key":"e_1_2_8_45_1","unstructured":"LeeF.The EMQ project \u2010 The Massively Scalable MQTT Broker for IoT and Mobile Applications.https:\/\/emqtt.io"},{"key":"e_1_2_8_46_1","unstructured":"ChristianG.HiveMQ Open Source Community.https:\/\/www.hivemq.com\/developers\/community\/"},{"key":"e_1_2_8_47_1","unstructured":"NicolasJ.HBMQTT.https:\/\/github.com\/beerfactory\/hbmqtt"},{"key":"e_1_2_8_48_1","unstructured":"MatteoC.Mosca \u2010 MQTT broker as a module.https:\/\/www.mosca.io\/"},{"key":"e_1_2_8_49_1","unstructured":"Emitter Studios.Emitter \u2010 Scalable Real\u2010Time Communication Accross Devices.https:\/\/emitter.io\/"},{"key":"e_1_2_8_50_1","unstructured":"GRID System.GRID Server Open Sourced.https:\/\/grids.systems\/2019\/08\/02\/grid-server-open-sourced"},{"key":"e_1_2_8_51_1","unstructured":"BanksA GuptaR.MQTT Version 3.1.1.https:\/\/docs.oasis-open.org\/mqtt\/mqtt\/v3.1.1\/os\/mqtt-v3.1.1-os.html"},{"key":"e_1_2_8_52_1","unstructured":"BanksA BriggsE BorgendaleK GuptaR.MQTT Version 5.0.https:\/\/docs.oasis-open.org\/mqtt\/mqtt\/v5.0\/cs02\/mqtt-v5.0-cs02.html"},{"key":"e_1_2_8_53_1","unstructured":"Bertrand\u2010MartinezE FeioP NascimentoV PinheiroB Abel\u00e9mA.A methodology for classification and evaluation of IoT brokers.9th Latin American Network Operations and Management Symposium (LANOMS).Niteroi Brazil;2019. ISSN: 978\u20103\u2010903176\u201023\u20102.http:\/\/dl.ifip.org\/db\/conf\/lanoms\/lanoms2019\/195912_1.pdf"}],"container-title":["International Journal of Network Management"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fnem.2115","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/nem.2115","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/nem.2115","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/am-pdf\/10.1002\/nem.2115","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/nem.2115","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,2]],"date-time":"2023-09-02T03:48:11Z","timestamp":1693626491000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/nem.2115"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,19]]},"references-count":52,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2021,5]]}},"alternative-id":["10.1002\/nem.2115"],"URL":"https:\/\/doi.org\/10.1002\/nem.2115","archive":["Portico"],"relation":{},"ISSN":["1055-7148","1099-1190"],"issn-type":[{"value":"1055-7148","type":"print"},{"value":"1099-1190","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,6,19]]},"assertion":[{"value":"2020-01-21","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-04-14","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-06-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}