{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,6,6]],"date-time":"2024-06-06T02:56:48Z","timestamp":1717642608807},"reference-count":56,"publisher":"Association for Computing Machinery (ACM)","issue":"2","funder":[{"DOI":"10.13039\/501100002855","name":"Ministry of Science and Technology of the People's Republic of China","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100002855","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Basic Research Program of Shenzhen"},{"DOI":"10.13039\/501100002920","name":"Research Grants Council, University Grants Committee, Hong Kong","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100002920","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Knowl. Discov. Data"],"published-print":{"date-parts":[[2013,7]]},"abstract":"Learning explanatory features across multiple related tasks, or MultiTask Feature Selection (MTFS), is an important problem in the applications of data mining, machine learning, and bioinformatics. Previous MTFS methods fulfill this task by batch-mode training. This makes them inefficient when data come sequentially or when the number of training data is so large that they cannot be loaded into the memory simultaneously. In order to tackle these problems, we propose a novel online learning framework to solve the MTFS problem. A main advantage of the online algorithm is its efficiency in both time complexity and memory cost. The weights of the MTFS models at each iteration can be updated by closed-form solutions based on the average of previous subgradients. This yields the worst-case bounds of the time complexity and memory cost at each iteration, both in the order ofO<\/jats:italic>(d<\/jats:italic>\u00d7Q<\/jats:italic>), whered<\/jats:italic>is the number of feature dimensions andQ<\/jats:italic>is the number of tasks. Moreover, we provide theoretical analysis for the average regret of the online learning algorithms, which also guarantees the convergence rate of the algorithms. Finally, we conduct detailed experiments to show the characteristics and merits of the online learning algorithms in solving several MTFS problems.<\/jats:p>","DOI":"10.1145\/2499907.2499909","type":"journal-article","created":{"date-parts":[[2013,7,30]],"date-time":"2013-07-30T13:35:22Z","timestamp":1375191322000},"page":"1-27","update-policy":"http:\/\/dx.doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":34,"title":["Efficient online learning for multitask feature selection"],"prefix":"10.1145","volume":"7","author":[{"given":"Haiqin","family":"Yang","sequence":"first","affiliation":[{"name":"The Chinese University of Hong Kong, Shenzhen, China"}]},{"given":"Michael R.","family":"Lyu","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Shenzhen, China"}]},{"given":"Irwin","family":"King","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Shenzhen, China"}]}],"member":"320","published-online":{"date-parts":[[2013,8,2]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Aaker D. A. Kumar V. and Day G. S. 2006. Marketing Research 9th Ed. Wiley. Aaker D. A. Kumar V. and Day G. S. 2006. Marketing Research 9 th Ed. Wiley."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.5555\/1046920.1194905"},{"key":"e_1_2_1_3_1","first-page":"41","article-title":"Multi-task feature learning","volume":"19","author":"Argyriou A.","year":"2006","unstructured":"Argyriou , A. , Evgeniou , T. , and Pontil , M. 2006 . Multi-task feature learning . Adv. Neural Inf. Process. Syst. 19 , 41 -- 48 . Argyriou, A., Evgeniou, T., and Pontil, M. 2006. Multi-task feature learning. Adv. Neural Inf. Process. Syst. 19, 41--48.","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-007-5040-8"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1645953.1646169"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1162\/153244304322765658"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.5555\/1390681.1390691"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-007-5043-5"},{"key":"e_1_2_1_9_1","volume-title":"Proceedings of the 16th Annual Conference on Learning Theory (COLT'03)","author":"Ben-David S.","unstructured":"Ben-David , S. and Schuller , R . 2003. Exploiting task relatedness for mulitple task learning . In Proceedings of the 16th Annual Conference on Learning Theory (COLT'03) . 567--580. Ben-David, S. and Schuller, R. 2003. Exploiting task relatedness for mulitple task learning. In Proceedings of the 16th Annual Conference on Learning Theory (COLT'03). 567--580."},{"key":"e_1_2_1_10_1","doi-asserted-by":"crossref","unstructured":"Boyd S. and Vandenberghe L. 2004. Convex Optimization. Cambridge University Press. Boyd S. and Vandenberghe L. 2004. Convex Optimization. Cambridge University Press.","DOI":"10.1017\/CBO9780511804441"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1007379606734"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1835804.1835952"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2086737.2086742"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553392"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2009.128"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/11776420_34"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.1953064"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.5555\/3121576.3121615"},{"key":"e_1_2_1_19_1","first-page":"2873","article-title":"Efficient learning using forward-backward splitting","volume":"10","author":"Duchi J.","year":"2009","unstructured":"Duchi , J. and Singer , Y. 2009 . Efficient learning using forward-backward splitting . J. Mach. Learn. Res. 10 , 2873 -- 2898 . Duchi, J. and Singer, Y. 2009. Efficient learning using forward-backward splitting. J. Mach. Learn. Res. 10, 2873--2898.","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.5555\/1046920.1088693"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1014052.1014067"},{"key":"e_1_2_1_22_1","unstructured":"Friedman J. Hastie T. and Tibshirani R. 2010. A note on the group lasso and a sparse group lasso. http:\/\/arxiv.org\/pdf\/1001.0736.pdf. Friedman J. Hastie T. and Tibshirani R. 2010. A note on the group lasso and a sparse group lasso. http:\/\/arxiv.org\/pdf\/1001.0736.pdf."},{"key":"e_1_2_1_23_1","volume-title":"Data Mining: Concepts and Techniques","author":"Han J.","year":"2006","unstructured":"Han , J. and Kamber , M . 2006 . Data Mining: Concepts and Techniques 2 nd Ed. Morgan Kaufmann , San Fransisco . Han, J. and Kamber, M. 2006. Data Mining: Concepts and Techniques 2nd Ed. Morgan Kaufmann, San Fransisco.","edition":"2"},{"key":"e_1_2_1_24_1","volume-title":"Proceedings of the 24th AAAI Conference on Artificial Intelligence.","author":"Han Y.","unstructured":"Han , Y. , Wu , F. , Jia , J. , Zhuang , Y. , and Yu , B . 2010. Multi-task sparse discriminant analysis (mtsda) with overlapping categories . In Proceedings of the 24th AAAI Conference on Artificial Intelligence. Han, Y., Wu, F., Jia, J., Zhuang, Y., and Yu, B. 2010. Multi-task sparse discriminant analysis (mtsda) with overlapping categories. In Proceedings of the 24th AAAI Conference on Artificial Intelligence."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-007-5016-8"},{"key":"e_1_2_1_26_1","first-page":"781","article-title":"Accelerated gradient methods for stochastic optimization and online learning","volume":"22","author":"Hu C.","year":"2009","unstructured":"Hu , C. , Kwok , J. , and Pan , W. 2009 . Accelerated gradient methods for stochastic optimization and online learning . Adv. Neural Inf. Process. Syst. 22 , 781 -- 789 . Hu, C., Kwok, J., and Pan, W. 2009. Accelerated gradient methods for stochastic optimization and online learning. Adv. Neural Inf. Process. Syst. 22, 781--789.","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015330.1015426"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.1953052"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.5555\/1577069.1577097"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1287\/mksc.15.2.173"},{"key":"e_1_2_1_31_1","volume-title":"Proceedings of the IEEE World Congress on Computational Intelligence (WCCI'12)","author":"Ling G.","unstructured":"Ling , G. , Yang , H. , King , I. , and Lyu , M. R . 2012. Online learning for collaborative filtering . In Proceedings of the IEEE World Congress on Computational Intelligence (WCCI'12) . 1--8. Ling, G., Yang, H., King, I., and Lyu, M. R. 2012. Online learning for collaborative filtering. In Proceedings of the IEEE World Congress on Computational Intelligence (WCCI'12). 1--8."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553458"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1557019.1557082"},{"key":"e_1_2_1_34_1","volume-title":"Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (UAI'09)","author":"Liu J.","unstructured":"Liu , J. , Ji , S. , and Ye , J . 2009c. Multi-task feature learning via efficient l2; 1 norm minimization . In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (UAI'09) . Liu, J., Ji, S., and Ye, J. 2009c. Multi-task feature learning via efficient l2; 1 norm minimization. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (UAI'09)."},{"key":"e_1_2_1_35_1","volume-title":"Slep: Sparse learning with efficient projections","author":"Liu J.","year":"2009","unstructured":"Liu , J. , Ji , S. , and Ye ., J. 2009 d. Slep: Sparse learning with efficient projections . http:\/\/www.public.asu.edu\/ye02\/Software\/SLEP. Liu, J., Ji, S., and Ye., J. 2009d. Slep: Sparse learning with efficient projections. http:\/\/www.public.asu.edu\/ye02\/Software\/SLEP."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-007-0149-x"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11222-008-9111-x"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1137\/090763184"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553484"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-007-5014-x"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/1273496.1273598"},{"key":"e_1_2_1_42_1","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the lasso","volume":"58","author":"Tibshirani R.","year":"1996","unstructured":"Tibshirani , R. 1996 . Regression shrinkage and selection via the lasso . J. Roy. Statist. Soc. B58 , 1, 267 -- 288 . Tibshirani, R. 1996. Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. B58, 1, 267--288.","journal-title":"J. Roy. Statist. Soc."},{"key":"e_1_2_1_43_1","volume-title":"The Nature of Statistical Learning Theory","author":"Vapnik V.","unstructured":"Vapnik , V. 1999. The Nature of Statistical Learning Theory 2 nd Ed. Springer , New York . Vapnik, V. 1999. The Nature of Statistical Learning Theory 2nd Ed. Springer, New York.","edition":"2"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.5555\/1756006.1953017"},{"key":"e_1_2_1_45_1","volume-title":"Proceedings of the 27th International Conference on Machine Learning (ICML'10)","author":"Xu Z.","unstructured":"Xu , Z. , Jin , R. , Yang , H. , King , I. , and Lyu , M. R . 2010. Simple and efficient multiple kernel learning by group lasso . In Proceedings of the 27th International Conference on Machine Learning (ICML'10) . 1175--1182. Xu, Z., Jin, R., Yang, H., King, I., and Lyu, M. R. 2010. Simple and efficient multiple kernel learning by group lasso. In Proceedings of the 27th International Conference on Machine Learning (ICML'10). 1175--1182."},{"key":"e_1_2_1_46_1","volume-title":"Proceedings of the International Joint Conference on Neural Networks (IJCNN'10)","author":"Yang H.","unstructured":"Yang , H. , King , I. , and Lyu , M. R . 2010a. Multi-task learning for one-class classification . In Proceedings of the International Joint Conference on Neural Networks (IJCNN'10) . 1--8. Yang, H., King, I., and Lyu, M. R. 2010a. Multi-task learning for one-class classification. In Proceedings of the International Joint Conference on Neural Networks (IJCNN'10). 1--8."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/1871437.1871706"},{"key":"e_1_2_1_48_1","unstructured":"Yang H. King I. and Lyu M. R. 2011a. Sparse Learning under Regularization Framework 1st Ed. Lambert Academic Publishing. Yang H. King I. and Lyu M. R. 2011a. Sparse Learning under Regularization Framework 1 st Ed. Lambert Academic Publishing."},{"key":"e_1_2_1_49_1","volume-title":"Proceedings of the 27th International Conference on Machine Learning (ICML'10)","author":"Yang H.","unstructured":"Yang , H. , Xu , Z. , King , I. , and Lyu , M. R . 2010c. Online learning for group lasso . In Proceedings of the 27th International Conference on Machine Learning (ICML'10) . 1191--1198. Yang, H., Xu, Z., King, I., and Lyu, M. R. 2010c. Online learning for group lasso. In Proceedings of the 27th International Conference on Machine Learning (ICML'10). 1191--1198."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2010.2103571"},{"key":"e_1_2_1_51_1","volume-title":"Proceedings of the 24th AAAI Conference on Artificial Intelligence (AAAI'10)","author":"Zhang Y.","year":"2010","unstructured":"Zhang , Y. 2010 . Multi-task active learning with output constraints . In Proceedings of the 24th AAAI Conference on Artificial Intelligence (AAAI'10) . Zhang, Y. 2010. Multi-task active learning with output constraints. In Proceedings of the 24th AAAI Conference on Artificial Intelligence (AAAI'10)."},{"key":"e_1_2_1_52_1","first-page":"2559","article-title":"Probabilistic multi-task feature selection","volume":"23","author":"Zhang Y.","year":"2010","unstructured":"Zhang , Y. , Yeung , D.-Y. , and Xu , Q. 2010 . Probabilistic multi-task feature selection . Adv. Neural Inf. Process. Syst. 23 , 2559 -- 2567 . Zhang, Y., Yeung, D.-Y., and Xu, Q. 2010. Probabilistic multi-task feature selection. Adv. Neural Inf. Process. Syst. 23, 2559--2567.","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.2021051"},{"key":"e_1_2_1_54_1","volume-title":"Proceedings of the 28th International Conference on Machine Learning (ICML'11)","author":"Zhao P.","unstructured":"Zhao , P. , Hoi , S. C. H. , Jin , R. , and Yang , T . 2011b. Online auc maximization . In Proceedings of the 28th International Conference on Machine Learning (ICML'11) . 233--240. Zhao, P., Hoi, S. C. H., Jin, R., and Yang, T. 2011b. Online auc maximization. In Proceedings of the 28th International Conference on Machine Learning (ICML'11). 233--240."},{"key":"e_1_2_1_55_1","volume-title":"Proceeding of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS'10)","author":"Zhou Y.","unstructured":"Zhou , Y. , Jin , R. , and Hoi , S. C . 2010. Exclusive lasso for multi-task feature selection . In Proceeding of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS'10) . Zhou, Y., Jin, R., and Hoi, S. C. 2010. Exclusive lasso for multi-task feature selection. In Proceeding of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS'10)."},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9868.2005.00503.x"}],"container-title":["ACM Transactions on Knowledge Discovery from Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2499907.2499909","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,15]],"date-time":"2024-05-15T18:01:41Z","timestamp":1715796101000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2499907.2499909"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2013,7]]},"references-count":56,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2013,7]]}},"alternative-id":["10.1145\/2499907.2499909"],"URL":"https:\/\/doi.org\/10.1145\/2499907.2499909","relation":{},"ISSN":["1556-4681","1556-472X"],"issn-type":[{"value":"1556-4681","type":"print"},{"value":"1556-472X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2013,7]]},"assertion":[{"value":"2011-01-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2012-11-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2013-08-02","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}