{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T11:07:58Z","timestamp":1740136078195,"version":"3.37.3"},"reference-count":39,"publisher":"Oxford University Press (OUP)","issue":"7","license":[{"start":{"date-parts":[[2022,4,9]],"date-time":"2022-04-09T00:00:00Z","timestamp":1649462400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"name":"National Institutes of Health Office of the Director: Data and Research Center","award":["5 U2C OD023196"]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,6,14]]},"abstract":"Abstract<\/jats:title>\n \n Objective<\/jats:title>\n A participant\u2019s medical history is important in clinical research and can be captured from electronic health records (EHRs) and self-reported surveys. Both can be incomplete, EHR due to documentation gaps or lack of interoperability and surveys due to recall bias or limited health literacy. This analysis compares medical history collected in the All of Us Research Program through both surveys and EHRs.<\/jats:p>\n <\/jats:sec>\n \n Materials and Methods<\/jats:title>\n The All of Us medical history survey includes self-report questionnaire that asks about diagnoses to over 150 medical conditions organized into 12 disease categories. In each category, we identified the 3 most and least frequent self-reported diagnoses and retrieved their analogues from EHRs. We calculated agreement scores and extracted participant demographic characteristics for each comparison set.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n The 4th All of Us dataset release includes data from 314\u00a0994 participants; 28.3% of whom completed medical history surveys, and 65.5% of whom had EHR data. Hearing and vision category within the survey had the highest number of responses, but the second lowest positive agreement with the EHR (0.21). The Infectious disease category had the lowest positive agreement (0.12). Cancer conditions had the highest positive agreement (0.45) between the 2 data sources.<\/jats:p>\n <\/jats:sec>\n \n Discussion and Conclusion<\/jats:title>\n Our study quantified the agreement of medical history between 2 sources\u2014EHRs and self-reported surveys. Conditions that are usually undocumented in EHRs had low agreement scores, demonstrating that survey data can supplement EHR data. Disagreement between EHR and survey can help identify possible missing records and guide researchers to adjust for biases.<\/jats:p>\n <\/jats:sec>","DOI":"10.1093\/jamia\/ocac046","type":"journal-article","created":{"date-parts":[[2022,3,23]],"date-time":"2022-03-23T20:11:58Z","timestamp":1648066318000},"page":"1131-1141","source":"Crossref","is-referenced-by-count":21,"title":["Comparing medical history data derived from electronic health records and survey answers in the All of Us<\/i> Research Program"],"prefix":"10.1093","volume":"29","author":[{"given":"Lina","family":"Sulieman","sequence":"first","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University Medical Center , Nashville, Tennessee, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1916-6521","authenticated-orcid":false,"given":"Robert M","family":"Cronin","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University Medical Center , Nashville, Tennessee, USA"},{"name":"Department of Medicine, The Ohio State University , Columbus, Ohio, USA"}]},{"given":"Robert J","family":"Carroll","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University Medical Center , Nashville, Tennessee, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9066-9431","authenticated-orcid":false,"given":"Karthik","family":"Natarajan","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University , New York, New York, USA"}]},{"given":"Kayla","family":"Marginean","sequence":"additional","affiliation":[{"name":"Vanderbilt Institute of Clinical and Translational Research, Vanderbilt University Medical Center , Nashville, Tennessee, USA"}]},{"given":"Brandy","family":"Mapes","sequence":"additional","affiliation":[{"name":"Vanderbilt Institute of Clinical and Translational Research, Vanderbilt University Medical Center , Nashville, Tennessee, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6302-0389","authenticated-orcid":false,"given":"Dan","family":"Roden","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University Medical Center , Nashville, Tennessee, USA"},{"name":"Department of Medicine, Vanderbilt University Medical Center , Nashville, Tennessee, USA"}]},{"given":"Paul","family":"Harris","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University Medical Center , Nashville, Tennessee, USA"},{"name":"Vanderbilt Institute of Clinical and Translational Research, Vanderbilt University Medical Center , Nashville, Tennessee, USA"}]},{"given":"Andrea","family":"Ramirez","sequence":"additional","affiliation":[{"name":"Department of Medicine, Vanderbilt University Medical Center , Nashville, Tennessee, USA"},{"name":"Office of data and analytics, All of Us Research Program, National Institutes of Health , Bethesda, Maryland, USA"}]}],"member":"286","published-online":{"date-parts":[[2022,4,9]]},"reference":[{"year":"2021","key":"2022061415534764800_ocac046-B1"},{"year":"2021","author":"Nichol","key":"2022061415534764800_ocac046-B2"},{"issue":"6","key":"2022061415534764800_ocac046-B3","doi-asserted-by":"crossref","first-page":"531","DOI":"10.1093\/ptj\/85.6.531","article-title":"Collecting health history information: the accuracy of a patient self-administered questionnaire in an orthopedic outpatient setting","volume":"85","author":"Boissonnault","year":"2005","journal-title":"Phys Ther"},{"issue":"1","key":"2022061415534764800_ocac046-B4","doi-asserted-by":"crossref","first-page":"38","DOI":"10.15265\/IY-2017-007","article-title":"Clinical data reuse or secondary use: current status and potential future progress","volume":"26","author":"Meystre","year":"2017","journal-title":"Yearb Med Inform"},{"key":"2022061415534764800_ocac046-B5","first-page":"604","article-title":"Identification of patients with family history of pancreatic cancer\u2014investigation of an NLP system portability","volume":"216","author":"Mehrabi","year":"2015","journal-title":"Stud Health Technol Inform"},{"year":"2014","author":"Pradhan","key":"2022061415534764800_ocac046-B6","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/s14-2007"},{"issue":"5","key":"2022061415534764800_ocac046-B7","doi-asserted-by":"crossref","first-page":"565","DOI":"10.1001\/jama.293.5.565","article-title":"Missing clinical information during primary care visits","volume":"293","author":"Smith","year":"2005","journal-title":"JAMA"},{"issue":"5","key":"2022061415534764800_ocac046-B8","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1136\/jamia.2010.003962","article-title":"Extracting medical information from narrative patient records: the case of medication-related information","volume":"17","author":"Del\u00e9ger","year":"2010","journal-title":"J Am Med Inform Assoc"},{"key":"2022061415534764800_ocac046-B9","doi-asserted-by":"crossref","DOI":"10.5888\/pcd15.170085","article-title":"Linking data from health surveys and electronic health records: a demonstration project in two Chicago health center clinics","volume":"15","author":"Wagaw","year":"2018","journal-title":"Prev Chronic Dis"},{"key":"2022061415534764800_ocac046-B10","first-page":"834","article-title":"How accurate is information that patients contribute to their Electronic Health Record?","volume":"2005","author":"Wuerdeman","year":"2005","journal-title":"AMIA Ann Symp Proc"},{"key":"2022061415534764800_ocac046-B11","doi-asserted-by":"crossref","first-page":"668","DOI":"10.1056\/NEJMsr1809937","article-title":"The \u201cAll of Us\u201d Research Program","volume":"381","author":"Denny","year":"2019","journal-title":"N Engl J Med"},{"issue":"7","key":"2022061415534764800_ocac046-B12","doi-asserted-by":"crossref","first-page":"e0234962","DOI":"10.1371\/journal.pone.0234962","article-title":"Diversity and inclusion for the All of Us research program: a scoping review","volume":"15","author":"Mapes","year":"2020","journal-title":"PLoS One"},{"year":"2020","author":"Ramirez","key":"2022061415534764800_ocac046-B13","doi-asserted-by":"publisher","DOI":"10.1101\/2020.05.29.20116905"},{"issue":"4","key":"2022061415534764800_ocac046-B14","doi-asserted-by":"crossref","first-page":"597","DOI":"10.1097\/EDE.0000000000001028","article-title":"Development of the initial surveys for the All of Us Research Program","volume":"30","author":"Cronin","year":"2019","journal-title":"Epidemiology"},{"year":"2021","key":"2022061415534764800_ocac046-B15"},{"issue":"5","key":"2022061415534764800_ocac046-B16","doi-asserted-by":"crossref","first-page":"516","DOI":"10.1197\/jamia.M2077","article-title":"Automating the assignment of diagnosis codes to patient encounters using example-based and machine learning techniques","volume":"13","author":"Pakhomov","year":"2006","journal-title":"J Am Med Inform Assoc"},{"issue":"2","key":"2022061415534764800_ocac046-B17","doi-asserted-by":"crossref","first-page":"203","DOI":"10.4065\/80.2.203","article-title":"Agreement between patient reports of cardiovascular disease and patient medical records","volume":"80","author":"St. Sauver","year":"2005","journal-title":"Mayo Clin Proc"},{"year":"2021","key":"2022061415534764800_ocac046-B18"},{"key":"2022061415534764800_ocac046-B19","first-page":"574","article-title":"Observational Health Data Sciences and Informatics (OHDSI): opportunities for observational researchers","volume":"216","author":"Hripcsak","year":"2015","journal-title":"Stud Health Technol Inform"},{"issue":"12","key":"2022061415534764800_ocac046-B20","doi-asserted-by":"crossref","first-page":"1618","DOI":"10.1093\/jamia\/ocy124","article-title":"Effect of vocabulary mapping for conditions on phenotype cohorts","volume":"25","author":"Hripcsak","year":"2018","journal-title":"J Am Med Inform Assoc"},{"key":"2022061415534764800_ocac046-B21","doi-asserted-by":"crossref","first-page":"250","DOI":"10.3389\/fgene.2014.00250","article-title":"Phenome-wide association studies demonstrating pleiotropy of genetic variants within FTO with and without adjustment for body mass index","volume":"5","author":"Cronin","year":"2014","journal-title":"Front Genet"},{"issue":"12","key":"2022061415534764800_ocac046-B22","doi-asserted-by":"crossref","first-page":"1102","DOI":"10.1038\/nbt.2749","article-title":"Systematic comparison of phenome-wide association study of electronic medical record data and genome-wide association study data","volume":"31","author":"Denny","year":"2013","journal-title":"Nat Biotechnol"},{"issue":"10","key":"2022061415534764800_ocac046-B23","doi-asserted-by":"crossref","first-page":"1359","DOI":"10.1093\/jamia\/ocy056","article-title":"PheProb: probabilistic phenotyping using diagnosis codes to improve power for genetic association studies","volume":"25","author":"Sinnott","year":"2018","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"2022061415534764800_ocac046-B24","doi-asserted-by":"crossref","first-page":"830","DOI":"10.1016\/j.jbi.2013.06.010","article-title":"Defining and measuring completeness of electronic health records for secondary use","volume":"46","author":"Weiskopf","year":"2013","journal-title":"J Biomed Inform"},{"key":"2022061415534764800_ocac046-B25","first-page":"1318","article-title":"Quantifying the longitudinal value of healthcare record collections for pharmacoepidemiology","volume":"2011","author":"Sperrin","year":"2011","journal-title":"AMIA Annu Symp Proc"},{"key":"2022061415534764800_ocac046-B26","article-title":"Agreement between patient-reported symptoms and their documentation in the medical record","volume":"14; 530\u20139","author":"Pakhomov","year":"2008","journal-title":"Am J Manag Care"},{"issue":"6","key":"2022061415534764800_ocac046-B27","doi-asserted-by":"crossref","first-page":"551","DOI":"10.1016\/0895-4356(90)90159-M","article-title":"High agreement but low kappa: II. Resolving the paradoxes","volume":"43","author":"Cicchetti","year":"1990","journal-title":"J Clin Epidemiol"},{"issue":"1","key":"2022061415534764800_ocac046-B28","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1177\/001316446002000104","article-title":"A coefficient of agreement for nominal scales","volume":"20","author":"Cohen","year":"1960","journal-title":"Educ Psychol Meas"},{"key":"2022061415534764800_ocac046-B29","doi-asserted-by":"crossref","first-page":"276","DOI":"10.11613\/BM.2012.031","article-title":"Interrater reliability: the kappa statistic","volume":"22","author":"McHugh","year":"2012","journal-title":"Biochem Med (Zagreb)"},{"key":"2022061415534764800_ocac046-B30","first-page":"912","article-title":"Accuracy and completeness of clinical coding using ICD-10 for ambulatory visits","volume":"2017","author":"Horsky","year":"2018","journal-title":"AMIA Annu Symp Proc"},{"key":"2022061415534764800_ocac046-B31","doi-asserted-by":"crossref","first-page":"e80","DOI":"10.1002\/cphg.80","article-title":"Using electronic health records to generate phenotypes for research","volume":"100","author":"Pendergrass","year":"2018","journal-title":"Curr Protoc Hum Genet"},{"key":"2022061415534764800_ocac046-B32","first-page":"423","article-title":"Parental functional health literacy relates to skip pattern questionnaire error and to child oral health","volume":"40","author":"Garrett","year":"2012","journal-title":"J Calif Dent Assoc"},{"key":"2022061415534764800_ocac046-B33","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1080\/10810730.2013.825668","article-title":"The relationship between health, education, and health literacy: results from the Dutch adult literacy and life skills survey","volume":"18 (Supp1","author":"van der Heide","year":"2013","journal-title":"J Health Commun"},{"issue":"1","key":"2022061415534764800_ocac046-B34","doi-asserted-by":"crossref","first-page":"394","DOI":"10.1186\/s12913-018-3197-4","article-title":"The role of health literacy in explaining the association between educational attainment and the use of out-of-hours primary care services in chronically ill people: a survey study","volume":"18","author":"Jansen","year":"2018","journal-title":"BMC Health Serv Res"},{"key":"2022061415534764800_ocac046-B35","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1016\/j.jbi.2016.08.005","article-title":"Computing disease incidence, prevalence and comorbidity from electronic medical records","volume":"63","author":"Bagley","year":"2016","journal-title":"J Biomed Inform"},{"key":"2022061415534764800_ocac046-B36","doi-asserted-by":"crossref","first-page":"101690","DOI":"10.1016\/j.canep.2020.101690","article-title":"Agreement between questionnaires and registry data on routes to diagnosis and milestone dates of the cancer diagnostic pathway","volume":"65","author":"Falborg","year":"2020","journal-title":"Cancer Epidemiol"},{"key":"2022061415534764800_ocac046-B37","doi-asserted-by":"crossref","first-page":"101741","DOI":"10.1016\/j.canep.2020.101741","article-title":"Agreement between questionnaires and registry data on routes to diagnosis and milestone dates of the cancer diagnostic pathway; methodological issues","volume":"67","author":"Sabour","year":"2020","journal-title":"Cancer Epidemiol"},{"key":"2022061415534764800_ocac046-B38","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1007\/s10552-018-1091-3","article-title":"Concordance of cancer registry and self-reported race, ethnicity, and cancer type: a report from the American Cancer Society\u2019s studies of cancer survivors","volume":"30","author":"Layne","year":"2019","journal-title":"Cancer Causes Control"},{"issue":"e1","key":"2022061415534764800_ocac046-B39","doi-asserted-by":"crossref","first-page":"e34","DOI":"10.1136\/amiajnl-2014-002909","article-title":"Assessment of administrative claims data for public health reporting of Salmonella in Tennessee","volume":"22","author":"Marder","year":"2015","journal-title":"J Am Med Inform Assoc"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/29\/7\/1131\/44062202\/ocac046.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/29\/7\/1131\/44062202\/ocac046.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,6,14]],"date-time":"2022-06-14T16:56:42Z","timestamp":1655225802000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/29\/7\/1131\/6565898"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,9]]},"references-count":39,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2022,4,9]]},"published-print":{"date-parts":[[2022,6,14]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocac046","relation":{},"ISSN":["1527-974X"],"issn-type":[{"type":"electronic","value":"1527-974X"}],"subject":[],"published-other":{"date-parts":[[2022,7,1]]},"published":{"date-parts":[[2022,4,9]]}}}