{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,3,27]],"date-time":"2025-03-27T12:13:34Z","timestamp":1743077614436,"version":"3.40.3"},"publisher-location":"Cham","reference-count":36,"publisher":"Springer Nature Switzerland","isbn-type":[{"type":"print","value":"9783031723803"},{"type":"electronic","value":"9783031723810"}],"license":[{"start":{"date-parts":[[2024,9,20]],"date-time":"2024-09-20T00:00:00Z","timestamp":1726790400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,9,20]],"date-time":"2024-09-20T00:00:00Z","timestamp":1726790400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025]]},"abstract":"Abstract<\/jats:title>In drug discovery, prioritizing compounds for testing is an important task. Active learning can assist in this endeavor by prioritizing molecules for label acquisition based on their estimated potential to enhance in-silico models. However, in specialized cases like toxicity modeling, limited dataset sizes can hinder effective training of modern neural networks for representation learning and to perform active learning. In this study, we leverage a transformer-based BERT model pretrained on millions of SMILES to perform active learning. Additionally, we explore different acquisition functions to assess their compatibility with pretrained BERT model. Our results demonstrate that pretrained models enhance active learning outcomes. Furthermore, we observe that active learning selects a higher proportion of positive compounds compared to random acquisition functions, an important advantage, especially in dealing with imbalanced toxicity datasets. Through a comparative analysis, we find that both BALD and EPIG acquisition functions outperform random acquisition, with EPIG exhibiting slightly superior performance over BALD. In summary, our study highlights the effectiveness of active learning in conjunction with pretrained models to tackle the problem of data scarcity.<\/jats:p>","DOI":"10.1007\/978-3-031-72381-0_12","type":"book-chapter","created":{"date-parts":[[2024,9,19]],"date-time":"2024-09-19T13:10:16Z","timestamp":1726751416000},"page":"149-159","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Deep Bayesian Experimental Design for\u00a0Drug Discovery"],"prefix":"10.1007","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9190-3023","authenticated-orcid":false,"given":"Muhammad Arslan","family":"Masood","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2392-0689","authenticated-orcid":false,"given":"Tianyu","family":"Cui","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1925-9154","authenticated-orcid":false,"given":"Samuel","family":"Kaski","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,9,20]]},"reference":[{"key":"12_CR1","doi-asserted-by":"publisher","unstructured":"Abd El Hafez, M.S., et al.: Characterization, in-silico, and in-vitro study of a new steroid derivative from Ophiocoma dentata as a potential treatment for COVID-19. Sci. Rep. 12(1), 5846 (2022). ISSN 2045-2322. https:\/\/doi.org\/10.1038\/s41598-022-09809-2, https:\/\/www.nature.com\/articles\/s41598-022-09809-2. Publisher: Nature Publishing Group","DOI":"10.1038\/s41598-022-09809-2"},{"key":"12_CR2","doi-asserted-by":"publisher","unstructured":"Bader, J., Narayanan, H., Arosio, P., Leroux, J.C.: Improving extracellular vesicles production through a Bayesian optimization-based experimental design. Eur. J. Pharm. Biopharm. 182, 103\u2013114 (2023). ISSN 0939-6411. https:\/\/doi.org\/10.1016\/j.ejpb.2022.12.004, https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0939641122002983","DOI":"10.1016\/j.ejpb.2022.12.004"},{"key":"12_CR3","doi-asserted-by":"publisher","unstructured":"BALLANTYNE, B.: Local and systemic ophthalmic pharmacology and toxicology of organophosphate and carbamate anticholinesterases. In: Toxicology of Organophosphate & Carbamate Compounds, pp. 423\u2013445. Elsevier, 2006. ISBN 978-0-12-088523-7. https:\/\/doi.org\/10.1016\/B978-012088523-7\/50032-6, https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/B9780120885237500326","DOI":"10.1016\/B978-012088523-7\/50032-6"},{"key":"12_CR4","doi-asserted-by":"publisher","unstructured":"Bemis, G.W., Murcko, M.A.: The properties of known drugs. 1. molecular frameworks. J. Med. Chem. 39(15), 2887\u20132893 (1996). ISSN 0022-2623. https:\/\/doi.org\/10.1021\/jm9602928. Publisher: American Chemical Society","DOI":"10.1021\/jm9602928"},{"issue":"518","key":"12_CR5","doi-asserted-by":"publisher","first-page":"859","DOI":"10.1080\/01621459.2017.1285773","volume":"112","author":"DM Blei","year":"2017","unstructured":"Blei, D.M., Kucukelbir, A., McAuliffe, J.D.: Variational inference: a review for statisticians. J. Am. Stat. Assoc. 112(518), 859\u2013877 (2017)","journal-title":"J. Am. Stat. Assoc."},{"key":"12_CR6","doi-asserted-by":"publisher","unstructured":"Choudhuri, S., Patton, G.W., Chanderbhan, R.F., Mattia, A., Klaassen, C.D.: From classical toxicology to Tox21: some critical conceptual and technological advances in the molecular understanding of the toxic response beginning from the last quarter of the 20th century. Toxicol. Sci. 161(1), 5\u201322 (2018). ISSN 1096-6080, 1096-0929. https:\/\/doi.org\/10.1093\/toxsci\/kfx186, https:\/\/academic.oup.com\/toxsci\/article\/161\/1\/5\/4102075","DOI":"10.1093\/toxsci\/kfx186"},{"key":"12_CR7","unstructured":"Cremaschi, A., Frigessi, A., Task\u00e9n, K., Zucknick, M.: A Bayesian approach to study synergistic interaction effects in in-vitro drug combination experiments. arXiv:1904.04901 (2019)"},{"key":"12_CR8","doi-asserted-by":"publisher","unstructured":"Daly, A.J., Stock, M., Baetens, J.M., De Baets, B.: Guiding mineralization co-culture discovery using bayesian optimization. Environ. Sci. Technol. 53(24), 14459\u201314469 (2019). ISSN 0013-936X. https:\/\/doi.org\/10.1021\/acs.est.9b05942. Publisher: American Chemical Society","DOI":"10.1021\/acs.est.9b05942"},{"key":"12_CR9","doi-asserted-by":"publisher","unstructured":"Demetriades, M., et al.: Interrogating and quantifying in vitro cancer drug pharmacodynamics via agent-based and bayesian monte carlo modelling. Pharmaceutics 14(4), 749 (2022). ISSN 1999-4923. https:\/\/doi.org\/10.3390\/pharmaceutics14040749, https:\/\/www.mdpi.com\/1999-4923\/14\/4\/749. Number: 4 Publisher: Multidisciplinary Digital Publishing Institute","DOI":"10.3390\/pharmaceutics14040749"},{"key":"12_CR10","unstructured":"Fabian, B., Edlich, T., Gaspar, H., Segler, M., Meyers, J., Fiscato, M.: Molecular representation learning with language models and domain-relevant auxiliary tasks. arXiv:2011.13230 (2020)"},{"key":"12_CR11","doi-asserted-by":"publisher","unstructured":"Festing, M.F.: Guidelines for the design and statistical analysis of experiments in papers submitted to ATLA. Alternatives to Laboratory Animals (2001). https:\/\/doi.org\/10.1177\/026119290102900409, https:\/\/journals.sagepub.com\/doi\/10.1177\/026119290102900409. Publisher: SAGE PublicationsSage UK: London, England","DOI":"10.1177\/026119290102900409"},{"key":"12_CR12","unstructured":"Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: Proceedings of the 34th International Conference on Machine Learning, pp. 1050\u20131059 (2016)"},{"key":"12_CR13","unstructured":"Gal, Y., Islam, R. and Ghahramani, Z.: Deep Bayesian active learning with image data. In: International Conference on Machine Learning, pp. 1183\u20131192. PMLR (2017)"},{"key":"12_CR14","doi-asserted-by":"crossref","unstructured":"Hao, Z., et al.: ASGN: an active semi-supervised graph neural network for molecular property prediction. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 731\u2013752 (2020)","DOI":"10.1145\/3394486.3403117"},{"key":"12_CR15","doi-asserted-by":"publisher","unstructured":"Hennessey, V.G., Rosner, G.L., Bast Jr, R.C., Chen, M.Y.: A Bayesian approach to dose-response assessment and synergy and its application to in vitro dose-response studies. Biometrics, 66(4), 1275\u20131283 (2010). ISSN 0006-341X. https:\/\/doi.org\/10.1111\/j.1541-0420.2010.01403.x","DOI":"10.1111\/j.1541-0420.2010.01403.x"},{"key":"12_CR16","unstructured":"Houlsby, N., Husz\u00e1r, F., Ghahramani, Z., Lengyel, M.: Bayesian active learning for classification and preference learning. arXiv preprint arXiv:1112.5745 (2011)"},{"key":"12_CR17","doi-asserted-by":"publisher","unstructured":"Johnston, S.T., Ross, J.V., Binder, B.J., McElwain, D.S., Haridas, P., Simpson, M.J.: Quantifying the effect of experimental design choices for in vitro scratch assays. J. Theor. Biol. 400, 19\u201331 (2016). ISSN 0022-5193. https:\/\/doi.org\/10.1016\/j.jtbi.2016.04.012, https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0022519316300406","DOI":"10.1016\/j.jtbi.2016.04.012"},{"key":"12_CR18","unstructured":"Kendall, A., Gal, Y.: What uncertainties do we need in Bayesian deep learning for computer vision? In: Advances in Neural Information Processing Systems, vol. 30 (2017)"},{"key":"12_CR19","doi-asserted-by":"publisher","unstructured":"Khan, A., et al.: Toward real-world automated antibody design with combinatorial Bayesian optimization. Cell Rep. Methods 3(1) (2023.) ISSN 2667-2375. https:\/\/doi.org\/10.1016\/j.crmeth.2022.100374, https:\/\/www.cell.com\/cell-reports-methods\/abstract\/S2667-2375(22)00276-4. Publisher: Elsevier","DOI":"10.1016\/j.crmeth.2022.100374"},{"key":"12_CR20","doi-asserted-by":"publisher","unstructured":"Merino-Casallo, F., Gomez-Benito, M.J., Juste-Lanas, Y., Martinez-Cantin, R., Garcia-Aznar, J.M.: Integration of in vitro and in silico models using bayesian optimization with an application to stochastic modeling of mesenchymal 3D cell migration. Front. Phys. 9, 1246 (2018). ISSN 1664-042X. https:\/\/doi.org\/10.3389\/fphys.2018.01246, https:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC6142046\/","DOI":"10.3389\/fphys.2018.01246"},{"issue":"6","key":"12_CR21","doi-asserted-by":"publisher","first-page":"547","DOI":"10.1007\/s11627-016-9786-1","volume":"52","author":"RP Niedz","year":"2016","unstructured":"Niedz, R.P., Evens, T.J.: Design of Experiments (DOE)\u2014history, concepts, and relevance to in vitro culture. Vitro Cell. Dev. Biol. Plant 52(6), 547\u2013562 (2016). https:\/\/doi.org\/10.1007\/s11627-016-9786-1","journal-title":"Vitro Cell. Dev. Biol. Plant"},{"key":"12_CR22","doi-asserted-by":"publisher","unstructured":"Pauwels, E., Lajaunie, C., Vert, J.P.: A Bayesian active learning strategy for sequential experimental design in systems biology. BMC Syst. Biol. 8(1), 102 (2014). ISSN 1752-0509. https:\/\/doi.org\/10.1186\/s12918-014-0102-6, https:\/\/bmcsystbiol.biomedcentral.com\/articles\/10.1186\/s12918-014-0102-6","DOI":"10.1186\/s12918-014-0102-6"},{"key":"12_CR23","doi-asserted-by":"publisher","unstructured":"Raies, A.B., Bajic, V.B.: In silico toxicology: computational methods for the prediction of chemical toxicity: computational methods for the prediction of chemical toxicity. Wiley Interdiscip. Rev. Comput. Mol. Sci. 6(2), 147\u2013172 (2016). ISSN 17590876. https:\/\/doi.org\/10.1002\/wcms.1240, https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/wcms.1240","DOI":"10.1002\/wcms.1240"},{"issue":"1","key":"12_CR24","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1214\/23-STS915","volume":"39","author":"T Rainforth","year":"2024","unstructured":"Rainforth, T., Foster, A., Ivanova, D.R., Bickford Smith, F.: Modern Bayesian experimental design. Stat. Sci. 39(1), 100\u2013114 (2024)","journal-title":"Stat. Sci."},{"key":"12_CR25","doi-asserted-by":"crossref","unstructured":"Rakesh, V., Jain, S.: Efficacy of Bayesian neural networks in active learning. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, pp. 2601\u20132609 (2021)","DOI":"10.1109\/CVPRW53098.2021.00294"},{"key":"12_CR26","doi-asserted-by":"publisher","unstructured":"Richard, A.M., et al.: The Tox21 10K compound library: collaborative chemistry advancing toxicology. Chem. Res. Toxicol. 34(2), 189\u2013216 (2021). ISSN 0893-228X, 1520-5010. https:\/\/doi.org\/10.1021\/acs.chemrestox.0c00264, https:\/\/pubs.acs.org\/doi\/10.1021\/acs.chemrestox.0c00264","DOI":"10.1021\/acs.chemrestox.0c00264"},{"key":"12_CR27","doi-asserted-by":"publisher","unstructured":"R\u00f8nneberg, L., Cremaschi, A., Hanes, R., Enserink, J.M., Zucknick, M.: Bayesynergy: flexible Bayesian modelling of synergistic interaction effects in in vitro drug combination experiments. Briefings Bioinform. 22(6), bbab251 (2021). ISSN 1477-4054. https:\/\/doi.org\/10.1093\/bib\/bbab251","DOI":"10.1093\/bib\/bbab251"},{"issue":"11","key":"12_CR28","doi-asserted-by":"publisher","first-page":"1243","DOI":"10.1007\/s40273-021-01065-y","volume":"39","author":"M Schlander","year":"2021","unstructured":"Schlander, M., Hernandez-Villafuerte, K., Cheng, C.-Y., Mestre-Ferrandiz, J., Baumann, M.: How much does it cost to research and develop a new drug? a systematic review and assessment. Pharmacoeconomics 39(11), 1243\u20131269 (2021). https:\/\/doi.org\/10.1007\/s40273-021-01065-y","journal-title":"Pharmacoeconomics"},{"key":"12_CR29","unstructured":"Smith, F.B., Kirsch, A., Farquhar, S., Gal, Y., Foster, A., Rainforth, T.: Prediction-oriented Bayesian active learning. In: International Conference on Artificial Intelligence and Statistics, pp. 7331\u20137348. PMLR (2023)"},{"key":"12_CR30","unstructured":"Smith, F.B., Kirsch, A., Farquhar, S., Gal, Y., Foster, A., Rainforth, T.: Prediction-oriented Bayesian active learning. arXiv:2304.08151v1 (2023)"},{"key":"12_CR31","unstructured":"Smith, F.B., Foster, A., Rainforth, T.: Making better use of unlabelled data in Bayesian active learning. In: International Conference on Artificial Intelligence and Statistics, pp. 847\u2013855. PMLR (2024)"},{"key":"12_CR32","doi-asserted-by":"publisher","unstructured":"Sun, D., Gao, W., Hu, H. and Zhou, S.: Why 90% of clinical drug development fails and how to improve it? Acta Pharmaceutica Sinica. B 12(7), 3049\u20133062 (2022). ISSN 2211-3835. https:\/\/doi.org\/10.1016\/j.apsb.2022.02.002, https:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC9293739\/","DOI":"10.1016\/j.apsb.2022.02.002"},{"key":"12_CR33","doi-asserted-by":"publisher","unstructured":"Tabernilla, A., et al.: In vitro liver toxicity testing of chemicals: a pragmatic approach. Int. J. Mol. Sci. 22(9), 5038 (2021). ISSN 1422-0067. https:\/\/doi.org\/10.3390\/ijms22095038, https:\/\/www.mdpi.com\/1422-0067\/22\/9\/5038","DOI":"10.3390\/ijms22095038"},{"key":"12_CR34","doi-asserted-by":"publisher","unstructured":"Van Norman, G.A.: Phase II trials in drug development and adaptive trial design. JACC: Basic Transl. Sci. 4(3), 428\u2013437 (2019). ISSN 2452302X. https:\/\/doi.org\/10.1016\/j.jacbts.2019.02.005, https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S2452302X19300658","DOI":"10.1016\/j.jacbts.2019.02.005"},{"issue":"35","key":"12_CR35","doi-asserted-by":"publisher","first-page":"8154","DOI":"10.1039\/C9SC00616H","volume":"10","author":"Y Zhang","year":"2019","unstructured":"Zhang, Y., et al.: Bayesian semi-supervised learning for uncertainty-calibrated prediction of molecular properties and active learning. Chem. Sci. 10(35), 8154\u20138163 (2019)","journal-title":"Chem. Sci."},{"key":"12_CR36","doi-asserted-by":"crossref","unstructured":"Zhou, Z., Kearnes, S., Li, L., Zare, R.N., Riley, P.: Optimization of molecules via deep reinforcement learning. Sci. Rep. 9(1), 10752 (2019)","DOI":"10.1038\/s41598-019-47148-x"}],"container-title":["Lecture Notes in Computer Science","AI in Drug Discovery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-031-72381-0_12","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,19]],"date-time":"2024-09-19T13:12:54Z","timestamp":1726751574000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-031-72381-0_12"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,20]]},"ISBN":["9783031723803","9783031723810"],"references-count":36,"URL":"https:\/\/doi.org\/10.1007\/978-3-031-72381-0_12","relation":{},"ISSN":["0302-9743","1611-3349"],"issn-type":[{"type":"print","value":"0302-9743"},{"type":"electronic","value":"1611-3349"}],"subject":[],"published":{"date-parts":[[2024,9,20]]},"assertion":[{"value":"20 September 2024","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"The authors have no competing interests to declare that\u00a0are relevant to the content of this article.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Disclosure of Interests"}},{"value":"AIDD","order":1,"name":"conference_acronym","label":"Conference Acronym","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"International Workshop on AI in Drug Discovery","order":2,"name":"conference_name","label":"Conference Name","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Lugano","order":3,"name":"conference_city","label":"Conference City","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Switzerland","order":4,"name":"conference_country","label":"Conference Country","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"2024","order":5,"name":"conference_year","label":"Conference Year","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"19 September 2024","order":7,"name":"conference_start_date","label":"Conference Start Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"19 September 2024","order":8,"name":"conference_end_date","label":"Conference End Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"1","order":9,"name":"conference_number","label":"Conference Number","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"aidd2024","order":10,"name":"conference_id","label":"Conference ID","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"https:\/\/e-nns.org\/wp-content\/uploads\/2024\/ICANN2024-AIDrugDis-CfP_final.pdf","order":11,"name":"conference_url","label":"Conference URL","group":{"name":"ConferenceInfo","label":"Conference Information"}}]}}