{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,27]],"date-time":"2024-08-27T13:13:47Z","timestamp":1724764427810},"reference-count":21,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,12,11]],"date-time":"2023-12-11T00:00:00Z","timestamp":1702252800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,12,11]],"date-time":"2023-12-11T00:00:00Z","timestamp":1702252800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Swiss Federal Institute of Technology Zurich"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Cheminform"],"abstract":"Abstract<\/jats:title>Time-split cross-validation is broadly recognized as the gold standard for validating predictive models intended for use in medicinal chemistry projects. Unfortunately this type of data is not broadly available outside of large pharmaceutical research organizations. Here we introduce the SIMPD (simulated medicinal chemistry project data) algorithm to split public data sets into training and test sets that mimic the differences observed in real-world medicinal chemistry project data sets. SIMPD uses a multi-objective genetic algorithm with objectives derived from an extensive analysis of the differences between early and late compounds in more than 130 lead-optimization projects run within the Novartis Institutes for BioMedical Research. Applying SIMPD to the real-world data sets produced training\/test splits which more accurately reflect the differences in properties and machine-learning performance observed for temporal splits than other standard approaches like random or neighbor splits. We applied the SIMPD algorithm to bioactivity data extracted from ChEMBL and created 99 public data sets which can be used for validating machine-learning models intended for use in the setting of a medicinal chemistry project. The SIMPD code and simulated data sets are available under open-source\/open-data licenses at github.com\/rinikerlab\/molecular_time_series.<\/jats:p>","DOI":"10.1186\/s13321-023-00787-9","type":"journal-article","created":{"date-parts":[[2023,12,11]],"date-time":"2023-12-11T11:02:22Z","timestamp":1702292542000},"update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["SIMPD: an algorithm for generating simulated time splits for validating machine learning approaches"],"prefix":"10.1186","volume":"15","author":[{"given":"Gregory A.","family":"Landrum","sequence":"first","affiliation":[]},{"given":"Maximilian","family":"Beckers","sequence":"additional","affiliation":[]},{"given":"Jessica","family":"Lanini","sequence":"additional","affiliation":[]},{"given":"Nadine","family":"Schneider","sequence":"additional","affiliation":[]},{"given":"Nikolaus","family":"Stiefl","sequence":"additional","affiliation":[]},{"given":"Sereina","family":"Riniker","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,12,11]]},"reference":[{"key":"787_CR1","doi-asserted-by":"publisher","first-page":"783","DOI":"10.1021\/ci400084k","volume":"53","author":"RP Sheridan","year":"2013","unstructured":"Sheridan RP (2013) Time-split cross-validation as a method for estimating the goodness of prospective prediction. J Chem Inf Model 53:783\u2013790","journal-title":"J Chem Inf Model"},{"key":"787_CR2","doi-asserted-by":"publisher","first-page":"945","DOI":"10.1093\/nar\/gkw1074","volume":"45","author":"A Gaulton","year":"2017","unstructured":"Gaulton A, Hersey A, Nowotka M, Bento AP, Chambers J, Mendez D, Mutowo P, Atkinson F, Bellis LJ, Cibri\u00e1n-Uhalte E, Davies M, Dedman N, Karlsson A, Magari\u00f1os MP, Overington JP, Papadatos G, Smit I, Leach AR (2017) The ChEMBL database in 2017. Nucleic Acids Res 45:945\u2013954","journal-title":"Nucleic Acids Res"},{"key":"787_CR3","doi-asserted-by":"publisher","first-page":"14425","DOI":"10.1021\/acs.jmedchem.0c01332","volume":"63","author":"A Schuffenhauer","year":"2020","unstructured":"Schuffenhauer A, Schneider N, Hintermann S, Auld D, Blank J, Cotesta S, Engeloch C, Fechner N, Gaul C, Giovannoni J, Jansen J, Joslin J, Krastel P, Lounkine E, Manchester J, Monovich LG, Pelliccioli AP, Schwarze M, Shultz MD, Stiefl N, Baeschlin DK (2020) Evolution of Novartis\u2019 small molecule screening deck design. J Med Chem 63:14425\u201314447","journal-title":"J Med Chem"},{"key":"787_CR4","doi-asserted-by":"publisher","first-page":"742","DOI":"10.1021\/ci100050t","volume":"50","author":"D Rogers","year":"2010","unstructured":"Rogers D, Hahn M (2010) Extended-connectivity fingerprints. J Chem Inf Model 50:742\u2013754","journal-title":"J Chem Inf Model"},{"key":"787_CR5","doi-asserted-by":"publisher","unstructured":"RDKit: Open Source Cheminformatics Toolkit, Version 2021.09.5. https:\/\/doi.org\/10.5281\/zenodo.6330241","DOI":"10.5281\/zenodo.6330241"},{"key":"787_CR6","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman L (2001) Random forests. Mach Learn 45:5\u201332","journal-title":"Mach Learn"},{"key":"787_CR7","unstructured":"scikit-learn: Machine Learning in Python. https:\/\/scikit-learn.org\/ Accessed 04 Jul 2022"},{"key":"787_CR8","doi-asserted-by":"publisher","first-page":"2623","DOI":"10.1021\/acs.jcim.1c00160","volume":"61","author":"C Esposito","year":"2021","unstructured":"Esposito C, Landrum GA, Schneider N, Stiefl N, Riniker S (2021) GHOST: adjusting the decision threshold to handle imbalanced data in machine learning. J Chem Inf Model 61:2623\u20132640","journal-title":"J Chem Inf Model"},{"key":"787_CR9","doi-asserted-by":"publisher","first-page":"428","DOI":"10.1038\/s41570-022-00391-9","volume":"6","author":"A Bender","year":"2022","unstructured":"Bender A, Schneider N, Segler M, Patrick Walters W, Engkvist O, Rodrigues T (2022) Evaluation guidelines for machine learning tools in the chemical sciences. Nat Rev Chem 6:428\u2013442","journal-title":"Nat Rev Chem"},{"key":"787_CR10","unstructured":"PREFER (2023). https:\/\/github.com\/rdkit\/PREFER Accessed 24 Feb 2023"},{"key":"787_CR11","unstructured":"SciPy. https:\/\/scipy.org\/ Accessed 04 Jul 2022"},{"key":"787_CR12","doi-asserted-by":"publisher","first-page":"704","DOI":"10.1021\/ci700099u","volume":"48","author":"SG Rohrer","year":"2008","unstructured":"Rohrer SG, Baumann K (2008) Impact of benchmark data set topology on the validation of virtual screening methods: Exploration and quantification by spatial statistics. J Chem Inf Model 48:704\u2013718","journal-title":"J Chem Inf Model"},{"key":"787_CR13","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1021\/ci00023a009","volume":"35","author":"R Taylor","year":"1995","unstructured":"Taylor R (1995) Simulation analysis of experimental design strategies for screening random compounds as potential new drugs and agrochemicals. J Chem Inf Comput Sci 35:59\u201367","journal-title":"J Chem Inf Comput Sci"},{"key":"787_CR14","first-page":"747","volume":"39","author":"D Butina","year":"1999","unstructured":"Butina D (1999) Unsupervised data base clustering based on Daylight\u2019s fingerprint and Tanimoto similarity: A fast and automated way to cluster small and large data sets. J Chem Inf Model 39:747\u2013750","journal-title":"J Chem Inf Model"},{"key":"787_CR15","volume-title":"Machine Learning","author":"TM Mitchell","year":"1997","unstructured":"Mitchell TM (1997) Machine Learning, 1st edn. McGraw-Hill Education, New York","edition":"1"},{"key":"787_CR16","doi-asserted-by":"publisher","first-page":"182","DOI":"10.1109\/4235.996017","volume":"6","author":"K Deb","year":"2002","unstructured":"Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: {NSGA-II}. IEEE Trans Evol Comput 6:182\u2013197","journal-title":"IEEE Trans Evol Comput"},{"key":"787_CR17","unstructured":"pymoo: Multi-objective Optimization in Python. https:\/\/pymoo.org\/ Accessed 04 Jul 2022"},{"key":"787_CR18","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1186\/s13321-018-0325-4","volume":"11","author":"N Bosc","year":"2019","unstructured":"Bosc N, Atkinson F, Felix E, Gaulton A, Hersey A, Leach AR (2019) Large scale comparison of QSAR and conformal prediction methods and their applications in drug discovery. J Cheminform 11:4","journal-title":"J Cheminform"},{"key":"787_CR19","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1186\/1758-2946-1-8","volume":"1","author":"P Ertl","year":"2009","unstructured":"Ertl P, Schuffenhauer A (2009) Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. J Cheminform 1:8","journal-title":"J Cheminform"},{"key":"787_CR20","doi-asserted-by":"publisher","first-page":"3714","DOI":"10.1021\/jm000942e","volume":"43","author":"P Ertl","year":"2000","unstructured":"Ertl P, Rohde B, Selzer P (2000) Fast calculation of molecular polar surface area as a sum of fragment-based contributions and its application to the prediction of drug transport properties. J Med Chem 43:3714\u20133717","journal-title":"J Med Chem"},{"key":"787_CR21","doi-asserted-by":"publisher","first-page":"6002","DOI":"10.1021\/acs.jcim.2c00785","volume":"62","author":"M Beckers","year":"2022","unstructured":"Beckers M, Fechner N, Stiefl N (2022) 25 years of small-molecule optimization at Novartis: a retrospective analysis of chemical series evolution. J Chem Inf Model 62:6002\u20136021","journal-title":"J Chem Inf Model"}],"container-title":["Journal of Cheminformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13321-023-00787-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13321-023-00787-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13321-023-00787-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,11]],"date-time":"2023-12-11T11:06:12Z","timestamp":1702292772000},"score":1,"resource":{"primary":{"URL":"https:\/\/jcheminf.biomedcentral.com\/articles\/10.1186\/s13321-023-00787-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,11]]},"references-count":21,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["787"],"URL":"https:\/\/doi.org\/10.1186\/s13321-023-00787-9","relation":{"has-preprint":[{"id-type":"doi","id":"10.26434\/chemrxiv-2023-x9pjf-v2","asserted-by":"object"}]},"ISSN":["1758-2946"],"issn-type":[{"value":"1758-2946","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,11]]},"assertion":[{"value":"1 September 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 November 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 December 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"119"}}