[PDF] Institutional differences in USMLE Step 1 and 2 CK performance

76841_7a31201891d84d7cfee811a00a1313378ffc1.pdf RESEARCHARTICLEInstitutionaldifferencesinUSMLEStep1and

2CKperformance:Cross-sectionalstudyof89

USallopathicmedicalschoolsJesseBurk-RafelID

1,RicardoW.Pulido2,YousefElfanagely3,JosephC.Kolars41DepartmentofInternalMedicine,NewYorkUniversityLangoneHealth,NewYork,NY,UnitedStatesof

America,2DepartmentofOtolaryngology-HeadandNeckSurgery,UniversityofWashington,Seattle,WA, UnitedStatesofAmerica,3DepartmentofInternalMedicine,BrownUniversity,Providence,RI,United StatesofAmerica,4DepartmentofInternalMedicine,UniversityofMichiganMedicalSchool,AnnArbor,MI,

UnitedStatesofAmerica

*jesse.rafel@nyulangone.orgAbstractIntroductionTheUnitedStatesMedicalLicensingExamination(USMLE)Step1andStep2Clinical

Knowledge(CK)areimportantfortraineemedicalknowledgeassessmentandlicensure, medicalschoolprogramassessment,andresidencyprogramapplicantscreening.Littleis knownabouthowUSMLEperformancevariesbetweeninstitutions.Thisobservational studyattemptstoidentifyinstitutionswithabove-predictedUSMLEperformance,whichmay

indicateeducationalprogramssuccessfulatpromotingstudents'medicalknowledge.MethodsSelf-reportedinstitution-leveldatawastabulatedfrompubliclyavailableUSNewsand

WorldReportandAssociationofAmericanMedicalCollegespublicationsfor131USallo- pathicmedicalschoolsfrom2012-2014.Bivariateandmultiplelinearregressionwereper- formed.TheprimaryoutcomewasinstitutionalmeanUSMLEStep1andStep2CKscores outsidea95%predictioninterval(�2standarddeviationsaboveorbelowpredicted)based

onmultipleregressionaccountingforstudents'prioracademicperformance.ResultsEighty-nineUSmedicalschools(54public,35private)reportedcompleteUSMLEscores

overthethree-yearstudyperiod,representingover39,000examinees.Institutionalmean gradepointaverage(GPA)andMedicalCollegeAdmissionTestscore(MCAT)achievedan adjustedR2of72%forStep1(standardizedMCAT0.7,GPA0.2)and41%forStep2CK (standardizedMCAT0.5,GPA0.3)inmultipleregression.Usingthisregressionmodel,5 institutionswereidentifiedwithabove-predictedinstitutionalUSMLEperformance,while3 institutionshadbelow-predictedperformance. PLOSONE|https://doi.org/10.1371/journal.pone.0224675November4,20191/10a1111111111 a1111111111 a1111111111 a1111111111 a1111111111

Citation:Burk-RafelJ,PulidoRW,ElfanagelyY,

KolarsJC(2019)Institutionaldifferencesin

USMLEStep1and2CKperformance:Cross-

sectionalstudyof89USallopathicmedical schools.PLoSONE14(11):e0224675.https://doi. org/10.1371/journal.pone.0224675

Editor:AndrewCarlMiller,EastCarolinaUniversity

BrodySchoolofMedicine,UNITEDSTATES

Received:July21,2019

Accepted:October19,2019

Published:November4,2019

PeerReviewHistory:PLOSrecognizesthe

benefitsoftransparencyinthepeerreview process;therefore,weenablethepublicationof allofthecontentofpeerreviewandauthor responsesalongsidefinal,publishedarticles.The editorialhistoryofthisarticleisavailablehere: https://doi.org/10.1371/journal.pone.0224675

Copyright:2019Burk-Rafeletal.Thisisanopen

accessarticledistributedunderthetermsofthe

CreativeCommonsAttributionLicense,which

permitsunrestricteduse,distribution,and reproductioninanymedium,providedtheoriginal authorandsourcearecredited.

DataAvailabilityStatement:Allrelevantdataare

withinthepaperanditsSupportingInformation files. ConclusionsThisexploratorystudyidentifiedseveralUSallopathicmedicalschoolswithsignificant above-orbelow-predictedUSMLEperformance.Althoughlimitedbyself-reporteddata,the findingsraisequestionsaboutinter-institutionalUSMLEperformanceparity,andthus,edu- cationalparity.Additionalworkisneededtodeterminetheetiologyandrobustnessofthe

observedperformancedifferences.IntroductionTheUnitedStatesMedicalLicensingExamination(USMLE)isa3-stepexaminationrequired

formedicallicensureintheUnitedStates.Thefirsttwoexams,USMLEStep1andStep2Clin- icalKnowledge(CK),assessmedicalstudents'masteryofbasicbiomedicalprinciplesandtheir clinicalapplications[1,2].About40,000traineestakeeachexamannually,ofwhichover35% arenon-US/Canadianmedicalstudents[3].Bothexamsarehigh-stakesparametersofmedical studentperformancecriticalforadvancement[4],residencyapplicantscreeningandselection [ 5 , 6 ],andfutureboardcertification[7].Multiplestudieshavedemonstratedcorrelations betweenindividualfactors-includingMedicalCollegeAdmissionTest(MCAT)score[8], undergraduategradepointaverage(GPA)[9],andstudybehaviors[10]-andUSMLEperfor- mance.However,littleisknownaboutinstitutionalUSMLEperformancevariation.One groupanalyzingdatafromthe1990sdemonstratedthatinstitutionalvariables,includingcur- riculardifferences,didnotpredictUSMLEperformance[11,12].Arecentstudyusingone yearofnationaldatafoundsomeevidenceofinter-institutionalUSMLEperformancediffer- ences,buttheshortstudydurationprecludesdefinitiveconclusions[13]. Inthisexploratory,institution-levelstudy,weanalyzeinstitutionalvariationinUSMLE Step1andStep2CKperformancerelativetomeanmatriculantGPAandMCAT.Ourprimary objectivewastoidentifyinstitutionswithabove-predictedUSMLEperformance,whichmay

indicateeducationalprogramssuccessfulatpromotingstudents'medicalknowledge.MethodsThisobservationalstudywasconductedinaccordancewiththeSTROBEguidelinesforobser-

vationalstudiesinepidemiology[14].DatasourcesWemanuallytabulatedself-reportedinstitutionaldata-aggregatepercentagesandmeansrep-

resentingyearlymedicalstudentcohortsatsingleinstitutions-fromtheannualUSNewsand WorldReport"BestGraduateSchools"publication(2008-2016editions)[15]andtheAssocia- tionofAmericanMedicalColleges(AAMC)MedicalSchoolAdmissionRequirementspublica- tion(2008-2012editions)[16]forall131USallopathicmedicalschools.Osteopathic institutionswereexcludedfromthisstudy,asosteopathicstudentstypicallytaketheCOMLEX licensingexaminationratherthantheUSMLEandveryfewUSosteopathicinstitutions reportedUSMLEperformancedata.Asamplesizecalculationwasnotperformedbecausewe obtainedavailabledataforacensusofUSallopathicmedicalschoolsduringthestudyperiod. Nationalaveragesforallallopathicmatriculantsandexamineeswereobtainedfromofficial AAMC[17]andUSMLEsources[18,19].InstitutionalReviewBoardapprovalwasnot requiredasnohumansubjectsoridentifiabledatawereinvolved.

InstitutionalUSMLEperformancedifferencesPLOSONE|https://doi.org/10.1371/journal.pone.0224675November4,20192/10Funding:Theauthorsreceivednospecificfunding

forthiswork.

Competinginterests:Wehavereadthejournal's

policyandtheauthorsofthismanuscripthavethe followingcompetinginterests:Dr.Burk-Rafel reportsworkingasaresearchconsultantfor

ScholarRx,adigitallearningplatformthatincludes

USMLEpreparationservices,duringthelatestages

ofwritingthismanuscript.ScholarRxwasnot involvedinthisstudyinanyway.Allotherauthors declarenocompetinginterests.Thisdoesnotalter ouradherencetoPLOSONEpoliciesonsharing dataandmaterials.

PrimaryoutcomemeasuresandpredictorvariablesTheprimaryoutcomemeasureswereinstitutionalmeanUSMLEStep1and2CKscores,aver-

agedoverthe3-yearstudyperiod2012-2014.Predictorvariablesincludedstudents"prioraca- demicperformance(institutionalmeanundergraduateGPAandMCAT,averagedover3 years)anddemographics(percentagenon-traditionalstudents,minoritystudents,undergradu- atebiologicalsciencesorhumanitiesmajors),andmedicalschoolfactors(acceptancerate,pub- lic/privatestatus,faculty-to-studentratio,NationalInstitutesofHealthresearchfunding, graduatesenteringprimarycare).MCATscoresrepresentedtotalscorescomputedasthesum oftheaverageinstitutionalscoresonall3sections(biologicalsciences,physicalsciences,verbal reasoning).InstitutionalUSMLEscoreswerematchedtoinstitutionalGPAandMCATaver- agesfromtwoorfouryearsprior(forStep1or2CK,respectively)toaccountforthetypical

lagbetweenmatriculationandUSMLEtesting.StatisticalanalysisAllanalysiswasattheinstitutionlevel.Weperformedordinaryleastsquareslinearregression

analysis,withtestofPearson'srforbivariatecorrelations.Conditionsoflinearity,nearlynor- malresiduals,andhomoscedasticitywerechecked[20].Institutionswith3-yearaverage USMLEperformanceoutsidea95%predictioninterval(regressionresidual�2standarddevi- ations,SD,frompredicted)wereidentified[21].Hypothesistestswere2-sidedwith=.05; ANOVAwasusedtoconfirmoverallsignificanceofmultipleregressions.Statisticalanalysis

wasdoneusingSPSSversion25.0(SPSSInc.,Chicago,Illinois).ResultsIntotal,89(54publicand35private)of131USallopathicmedicalschoolsreportedcomplete

USMLEscoresoverthe3-yearstudyperiod(68%reportingrate),representing39,615and

39,252Step1and2CKexaminees,respectively.Amongreportinginstitutions,theinstitu-

tionalmeanUSMLEStep1scorewas229.7(SD5.5)andStep2CKscorewas238.3(SD4.7) ( Table1).GPAandMCATscoresshowedminimalheterogeneityacrossthestudyyears(data notshown).USMLEscoresincreasedacrossthestudyyears,whichwasalsoobservednation- ally.TheaverageGPA,MCATscores,andUSMLEStep1scoresforthe89reportinginstitu- tionswereslightlyhigherthannationalaveragesforallmatriculants/examinees.Complete GPA,MCAT,andUSMLEdataforreportinginstitutionsandnationallyareprovidedinS1 Table

.PredictorsofinstitutionalUSMLEperformanceThestrongestpredictorofinstitutionalUSMLEscoreswaspriorstudentacademicperfor-

mance,includingundergraduateGPA(Step1,Pearson'sr=.64;Step2CK,r=.53;bothP< .001)andMCATscore(Step1,r=.84;Step2CK,r=.62;bothP<.001).Numerousstudent bodydemographicandinstitutionalfactorshadmoderatelystrongcorrelationswithinstitu- tionalUSMLEscoresinbivariateregression;however,whencontrollingforGPAandMCAT, thesecorrelationswereweakandnolongersignificant(Table2).Forexample,privateinstitu- tionswerecorrelatedwithhigherUSMLEStep1scores(r=.51,P<.001),butthiscorrelation vanishedaftercontrollingforGPAandMCAT(r=.12,P=.42),asprivateinstitutionsrecruit studentswithhigherMCATscorescomparedtopublicinstitutions(mean33.5vs.30.9,differ- ence2.7,95%CI1.9-3.5;P<.001). ThefinalregressionmodelutilizingGPAandMCATachievedanadjustedR2of72%for Step1(standardizedMCAT0.7,GPA0.2,modelP<.001)and41%forStep2CK

InstitutionalUSMLEperformancedifferencesPLOSONE|https://doi.org/10.1371/journal.pone.0224675November4,20193/10

(standardizedMCAT0.5,GPA0.3,modelP<.001).GPAaddedsignificantbutincremental validityevidenceoverMCATalone(Step1,R22%,P=.009;Step2CK,R24%,P=.02); accordingly,forvisualization,institutionalUSMLEwasregressedonMCATscorealone ( Fig1).Table1.AverageGPA,MCAT,andUSMLEStep1and2CKscoreamong89USallopathicmedicalschoolsandnationally.

AllSchools(n=89)

Average(SD)Public(n=54)

Average(SD)Private(n=35)

Average(SD)Pvalue�NationalAverage

GPA2010-20123.70(0.08)3.68(0.07)3.73(0.08)ns3.67

MCAT2010-201231.9(2.2)30.9(1.7)33.5(2.0)<.00131.1

USMLEStep1

2012227.6(6.1)225.4(5.3)230.9(5.6)<.001227

2013230.4(5.8)228.1(4.9)233.9(5.3)<.001228

2014231.1(5.6)229.0(4.6)234.4(5.6)<.001229

2012-2014(combined)229.7(5.5)227.5(4.4)233.1(5.2)<.001228.0

USMLEStep2CK

2012235.6(5.5)234.7(5.1)237.0(5.8)ns237

2013238.8(5.3)237.6(4.8)240.7(5.5)<.01238

2014240.5(4.5)239.3(4.1)242.4(4.4)<.01240

2012-2014(combined)238.3(4.7)237.2(4.2)240.0(4.9)<.01238.3

GPA,UndergraduateGradePointAverage;MCAT,MedicalCollegeAdmissionTestscore;USMLE,USMedicalLicensingExamination;CK,ClinicalKnowledge;ns,

notsignificantatP<.05threshold�Two-tailedt-testcomparingpublictoprivate

https://doi.org/10.1371/journal.pone.0224675.t001Table2.LinearregressionbetweenvariousinstitutionalcharacteristicsandinstitutionalUSMLEperformance,withoutandwithcontrolforaverageinstitutional

GPAandMCAT.

USMLEStep1Pearson's

rPartial

ρ†USMLEStep2CKPearson's

rPartial ρ‡InstitutionalGPA.64��-InstitutionalGPA.53��- InstitutionalMCAT.84��-InstitutionalMCAT.62��- USMLEStep2CK.56��.05USMLEStep1.56��.06 MinorityStudents.46��.16MinorityStudents.25�-.03 BiologicalScienceMajors-.36��-.07BiologicalScienceMajors-.27�-.12

HumanitiesMajors.13.07HumanitiesMajors.10.11

Non-TraditionalStudents.01-.13Non-TraditionalStudents-.03-.07 AcceptanceRate-.30��-.14AcceptanceRate-.17-.05 PrivateInstitution.51��.12PrivateInstitution.30��-.06 Faculty:StudentRatio.44��.01Faculty:StudentRatio.35��.06 NIHFunding.58��-.13NIHFunding.47��-.01 PrimaryCareGrads-.31��-.12PrimaryCareGrads-.10.17

GPA,UndergraduateGradePointAverage;MCAT,MedicalCollegeAdmissionTestscore;USMLE,USMedicalLicensingExamination;CK,ClinicalKnowledge;NIH,

NationalInstitutesofHealth.�P<.05��P<.01†PartialcorrelationcontrollingforGPAandMCAT(2010-12)‡PartialcorrelationcontrollingforGPAandMCAT(2008-10)

https://doi.org/10.1371/journal.pone.0224675.t002

InstitutionalUSMLEperformancedifferencesPLOSONE|https://doi.org/10.1371/journal.pone.0224675November4,20194/10

InstitutionalUSMLEperformancedifferencesPLOSONE|https://doi.org/10.1371/journal.pone.0224675November4,20195/10

Institutionswithabove-orbelow-predictedUSMLEperformanceUsingtheGPAandMCATregressionmodel,weidentifiedasubsetofinstitutionswith3-year

averageinstitutionalUSMLEscoresstatisticallyaboveorbelowpredicted(Table3).DiscussionInthisexploratorystudyof89USallopathicmedicalschools,weidentified5institutionswith

above-predictedinstitutionalUSMLEperformancebasedonthedescribedmodel.Theetiol- ogyoftheseinstitutions'relativesuccess(orthe3unnamedinstitutions'below-predictedper- formance)isunclear;wecanonlysaythatnumerousdemographicandinstitutionalfactorswe assesseddidnotaccountforthisvariation.Wehypothesizethatunmeasuredstudentfactors thatvarysystematicallybetweeninstitutions(e.g.,throughadmissionsprocesses)orinstitu- tion-specificfactors(e.g.,alignmentofcurriculawithUSMLEcontent)mayexplainthese institutionaldifferences.Forexample,medicalschoolsthatprovidecommerciallyavailable Step1questionbanks[22]orwherestudentstakeStep1afterthecoreclerkships[23]have reportedimprovedinstitutionalscores,demonstratingthatuniqueinstitutionalstrategiescan promotestudents'USMLEsuccess.Furtherstudyisneededtounderstandifthe5institutions identifiedherehaveuniquefactorsthatpromotedtheirstudents'successontheseexams. Wefoundthatinstitutions'averagestudentGPAandMCATaccountedforsubstantialvari- ationininstitutionalaverageUSMLEStep1andStep2CKscores,whichwasexpectedbased onpriorstudiesattheindividual[8,9]andinstitutionallevel[9,11,12].Importantly,institu- tionaldemographicfactors(suchaspercentminoritystudentsorbiologicalsciencesmajors) werecorrelatedwithinstitutionalUSMLEperformanceinbivariateregressionbutwerenot significantaftercontrollingforGPAandMCAT.NationalInstitutesofHealthresearchfund- ing,whichhadbeenpreviouslyshowntocorrelatewithinstitutionalUSMLEperformance [ 13 ],wassimilarlynotsignificantwhencontrollingforGPAandMCAT. Institutionswithapropensityformatchingstudentsintheprimarycarespecialtiesfamily medicine,pediatrics,andinternalmedicine-whichhavelowerUSMLEscreeningthresholds forresidencyinterviewsthanothermore"competitive"specialties[24]-tendedtorecruitstu- dentswithlowerGPAandMCATscores,andthuslowerinstitutionalUSMLEscores.Aswith otherinstitutionalfactors,however,institutions'primarycarespecialtyratewasnotassociated withdifferentialUSMLEperformancebeyonditsassociationwithGPAandMCAT. Suchfindingshighlightthecriticalimportanceofcontrollingforprioracademicperfor- mancewhenattemptingtoexplainUSMLEperformancedifferences.However,wedoubtthat pre-medicalstudents-akeyconsumeroftheannualUSNewsandWorldReportdata-consider thesecovariateswheninterpretinginstitutionalUSMLEscoresandidentifyingmedicalschools ofinterest.Indeed,undergraduatesmightconclude(erroneously)thatprivatemedicalschools outperformpublicschoolsontheUSMLE,wheninfactstudentsattendingprivateschools havehighertestscoresatmatriculation.Theremaybearoleforbettercontextualizingthis

datasothatpre-medicalstudentscanbeinformedconsumers.TheNationalBoardofMedicalFig1.RegressionanalysisofinstitutionalMCATversusUSMLEperformance.(A)Regressionanalysisofinstitutional

averagematriculantMedicalCollegeAdmissionTest(MCAT)score(2010-2012)versusinstitutionalaverageUSMedical

LicensingExamination(USMLE)Step1score(2012-2014)acrossn=89USallopathicmedicalschools,representing39,615

examinees.(B)RegressionanalysisofinstitutionalaveragematriculantMCATscore(2008-2010)versusinstitutionalaverage

USMLEStep2ClinicalKnowledge(CK)score(2012-2014)acrossn=89USallopathicmedicalschools,representing39,252

examinees.Forbothplots,eachbubblerepresents3-yearaverageatoneinstitution,withbubblesizereflectingnumber

examinedateachinstitution.Ordinaryleastsquaresbestfitline(solid)and95%predictioninterval(dashedlines)areshown,

withcoloreddatapointshighlightinginstitutionsoutsidethepredictioninterval. https://doi.org/10.1371/journal.pone.0224675.g001

InstitutionalUSMLEperformancedifferencesPLOSONE|https://doi.org/10.1371/journal.pone.0224675November4,20196/10

Examiners(NBME),whoproducetheUSMLE,arepositionedtomorerigorouslyexplorethe

relationshipbetweeninstitutionsandexamperformance.LimitationsThisstudyreliedonself-reportedinstitutionaldataviaathird-partypublication,astheNBME

doesnotpublishinstitutionalscoreperformance.Misreportingispossible,althoughwevali- datedthereportedscoresfromseveralinstitutions.USNewsandWorldReportprovidestheir methodologyfordatacollectionwitheachannualrelease[25],butdonotstatespecificsrelated tohowdataisvalidatedorstandardizedwithin-orbetween-schools.Forexample,itisunclear ifinstitutionshavediscretioninhowtheyformulatetheirinstitutionalMCATaverage,includ- inghowindividualswithmultipletestresultsarehandled,whichcanintroducebiasintothe relationshipbetweenMCATandUSMLEperformance[26]. Althoughweassessednumerousstudentandmedicalschoolfactors,somepotentially importantcovariates-suchaspercentofstudentswithadvanceddegrees,curricularstructure, timingofUSMLEexaminations,andschoolage-werenotincorporatedintothisstudybutare importantareasforfutureinvestigation.Forexample,someinstitutionshavemovedthe USMLEStep1testwindowtoaftercoreclinicalclerkships[23],withsmallbenefitsinscores andreducedfailurerates[27]. Moreover,only89of131USallopathicmedicalschools(68%)reportedcompletedata; non-reportersmaydifferinimportantways.Wefoundthatreportinginstitutions,ascom- paredtoanaverageofallstudentsnationally,hadslightlyhigheraverageGPAandMCAT scores,withanassociated1.5-pointhigheraverageUSMLEStep1score.Statisticalcompari- sonsofthesedifferencesarenotadvisablegiventhedifferentunitsofreporting(institutions vs.individuals);yettheverysmalldifferencessuggestthatthereportinginstitutionswere nationallyrepresentative.Therelativelyshort3-yearstudyperioddoesnotprecludethatthe observedinstitutionaloutliersmayrepresentrandomvariation;replicationwithlongerobser-

vationisneeded.Finally,ourstudywasecological;noinferencecanbemadethatinstitution-Table3.USallopathicmedicalschoolswithabove-orbelow-predictedainstitutionalUSMLEStep1orStep2CKperformance,2012-2014.

USMLEStep1

InstitutionAverageScore(SD)ScoreDeviationfromPredicted,PointsStandardizedResidual,SDExaminees,No.

UniversityofHawaii-Manoa234(3.2)+8.4+2.9182

UniversityofMissouri236(4.6)+8.2+2.8296

BaylorCollegeofMedicine241(1.0)+5.9+2.0517

InstitutionXb220(3.2)-5.9-2.0504

USMLEStep2CK

InstitutionAverageScore(SD)ScoreDeviationfromPredicted,PointsStandardizedResidual,SDExaminees,No.

EmoryUniversity250(2.6)+10.1+2.8424

UniversityofVirginia248(2.1)+7.1+2.0449

InstitutionX228(5.3)-7.3-2.0481

InstitutionY228(9.7)-9.2-2.6507

InstitutionZ230(3.2)-12.0-3.4305

USMLE,USMedicalLicensingExamination;CK,ClinicalKnowledge;SD,standarddeviation.aBasedonregressionmodelsincorporatinginstitutionalaverageMedicalCollegeAdmissionTest(MCAT)scoreandundergraduategradepointaverage(GPA)of

enteringstudents,asfollows:InstitutionalUSMLEStep1score=122+1.7�MCAT+14.1�GPA;InstitutionalUSMLEStep2CKscore=149+1.0�MCAT+15.6�GPA.bThenamesofinstitutionswithbelow-predictedinstitutionalUSMLEperformancewerewithheldduetothesensitiveandexploratorynatureofthisdata.

https://doi.org/10.1371/journal.pone.0224675.t003

InstitutionalUSMLEperformancedifferencesPLOSONE|https://doi.org/10.1371/journal.pone.0224675November4,20197/10

levelfindingstranslatetoindividualstudents(i.e.,theecologicalfallacy),andindeedonlyinsti- tutionalaveragesandcountswerereported(withoutanymeasureofstudent-to-studentvari-

ability).Nevertheless,thepurposeofthisstudywasonlytocompareinstitutions.ConclusionsWefoundthatinstitutionalaverageGPAandMCATscorescorrelatestronglywithinstitu-

tionalUSMLEperformance.Numerousstudentdemographicandinstitutionalfactorswere insignificantwhencontrollingforGPAandMCAT.Weidentifiedseveralinstitutionswithsig- nificantabove-orbelow-predictedUSMLEperformance,raisingquestionsaboutinter-institu- tionalUSMLEperformanceparity.Methodstoassessinstitutions'overallperformanceon knowledge-basedexamsmayofferaparametertoevaluatemedicalschoolsandtheircurricula, whileprovidingprospectivestudentswithvaluabledataregardingthesehigh-stakesexams. Additionalstudyisneededtoexploretheetiologyanddurabilityoftheobservedperformance differences,andtoincorporateotherstudentandinstitutionalfactorsthatmaybeimportant

predictorsofperformance.SupportinginformationS1Table.AverageGPA,MCAT,andUSMLEStep1and2CKfor89USallopathicmedical

schoolsreportingcompletedatafrom2012-2014,withassociatednationalaverages. (XLSX)AcknowledgmentsThisworkwasinspiredbyananonymouspostonMarch13,2013,intheAnastomosedblog titled:"Valueaddedintopmedicalschools?MCAT/GPAaspredictorsofUSMLEscores"(see https://anastomosed.wordpress.com/).JoelPurkiss,PhD(BaylorCollegeofMedicine,Hous- ton,Texas,USA)providedearlyguidanceonthedesignofthiswork.MartinV.Pusic,MD, MA(NewYorkUniversityLangoneHealth,NewYork,USA)andKentHecker,PhD(Univer- sityofCalgary,Alberta,Canada)providedhelpfulmethodologicadviceandmanuscript critiques.AuthorContributionsConceptualization:JesseBurk-Rafel,JosephC.Kolars.

Datacuration:JesseBurk-Rafel.

Formalanalysis:JesseBurk-Rafel,RicardoW.Pulido,YousefElfanagely.

Investigation:JesseBurk-Rafel.

Methodology:JesseBurk-Rafel,JosephC.Kolars.

Supervision:JosephC.Kolars.

Validation:JesseBurk-Rafel.

Visualization:JesseBurk-Rafel,RicardoW.Pulido,YousefElfanagely. Writing-originaldraft:JesseBurk-Rafel,RicardoW.Pulido,YousefElfanagely. Writing-review&editing:JesseBurk-Rafel,RicardoW.Pulido,YousefElfanagely,Joseph

C.Kolars.

InstitutionalUSMLEperformancedifferencesPLOSONE|https://doi.org/10.1371/journal.pone.0224675November4,20198/10

References1.HaistSA,KatsufrakisPJ,DillonGF.TheevolutionoftheUnitedStatesMedicalLicensingExamination (USMLE):Enhancingassessmentofpractice-relatedcompetencies.JAMA.2013;310(21):2245-

2246.https://doi.org/10.1001/jama.2013.282328PMID:24302081

2.USMedicalLicensingExamination,2019BulletinofInformation.NationalBoardofMedicalExaminers.

[cited2018Sep15].Availablefrom:https://www.usmle.org/pdfs/bulletin/2019bulletin.pdf.

3.USMedicalLicensingExamination,2018PerformanceData.NationalBoardofMedicalExaminers.

[cited2019Jul17].Availablefrom:https://www.usmle.org/performance-data/default.aspx#2018_step-1

4.AndrioleDA,JeffeDB.Prematriculationvariablesassociatedwithsuboptimaloutcomesforthe1994-

1999cohortofUSmedicalschoolmatriculants.JAMA.2010;304(11):1212-1219.https://doi.org/10.

1001/jama.2010.1321PMID:20841535

5.NationalResidentMatchingProgram,DataReleaseandResearchCommittee:Resultsofthe2018

NRMPProgramDirectorSurvey.NationalResidentMatchingProgram.Washington,DC.[cited2018 Sep15].Availablefrom:http://www.nrmp.org/wp-content/uploads/2018/07/NRMP-2018-Program-

Director-Survey-for-WWW.pdf

6.JayakumarKL,LipoffJB.USMedicalLicensingExaminationStep2ClinicalKnowledgescoreinderma-

tologyresidentselection.JAMADermatol.2017;153(11):1193-1194.https://doi.org/10.1001/ jamadermatol.2017.2924PMID:28832874

7.JeffeDB,AndrioleDA.FactorsassociatedwithAmericanBoardofMedicalSpecialtiesmemberboard

certificationamongUSmedicalschoolgraduates.JAMA.2011;306(9):961-970.https://doi.org/10.

1001/jama.2011.1099PMID:21900136

8.DonnonT,PaolucciEO,ViolatoC.ThepredictivevalidityoftheMCATformedicalschoolperformance

andmedicalboardlicensingexaminations:Ameta-analysisofthepublishedresearch.AcadMed.

2007;82(1):100-106.https://doi.org/10.1097/01.ACM.0000249878.25186.b7PMID:17198300

9.BascoWTJr,WayDP,GilbertGE,HudsonA.UndergraduateinstitutionalMCATscoresaspredictors

ofUSMLEStep1performance.AcadMed.2002;77(10S):13-16.

10.Burk-RafelJ,SantenSA,PurkissJ.StudybehaviorsandUSMLEStep1performance:Implicationsofa

studentself-directedparallelcurriculum.AcadMed.2017;92(11S):67-74.

11.HeckerK,ViolatoC.Howmuchdodifferencesinmedicalschoolsinfluencestudentperformance?A

longitudinalstudyemployinghierarchicallinearmodeling.TeachLearnMed.2008;20(2):104-113. https://doi.org/10.1080/10401330801991915PMID:18444195

12.HeckerK,ViolatoC.Medicalschoolcurricula:Docurricularapproachesaffectcompetenceinmedi-

cine?FamMed.2010;41(6):420-426.

13.Ghaffari-RafiA,LeeRE,FangR,MilesDJ.MultivariableanalysisoffactorsassociatedwithUSMLE

scoresacrossUSmedicalschools.BMCMedEd.2019;19:154.

14.vonElmE,AltmanDG,EggerM,PocockSJ,GotzschePC,VandenbrouckeJP.TheStrengtheningthe

ReportingofObservationalStudiesinEpidemiology(STROBE)Statement:Guidelinesforreporting observationalstudies.PLOSMed.2007;4(10):e296.https://doi.org/10.1371/journal.pmed.0040296

PMID:17941714

15.BestMedicalSchools:Research.USNewsandWorldReport.2008-2016editions,onlineandprint.

[cited2018Sep15].Availablefrom:http://grad-schools.usnews.rankingsandreviews.com/best- graduate-schools/top-medical-schools/research-rankings/Subscriptionrequired.

16.MedicalSchoolAdmissionRequirements.AssociationofAmericanMedicalColleges.Washington

(DC):2008-2012editions.

17.FACTSTable17.AssociationofAmericanMedicalColleges.2013.[cited2016Nov6]Availableby

requestfrom:https://www.aamc.org/data-reports/students-residents/report/facts

18.USMedicalLicensingExamination,PerformanceData.NationalBoardofMedicalExaminers.2014-

2016.Availablefrom:https://www.usmle.org/performance-data/

19.USMLEScoreInterpretationGuidelines.2015ed.Availablefrom:https://www.usmle.org/pdfs/

transcripts/USMLE_Step_Examination_Score_Interpretation_Guidelines.pdf

20.AltmanN,Krzywinski.Regressiondiagnostics.NatMeth.2016;13:385-386.

21.WernerJ,GriebelerEM.Newinsightsintonon-aviandinosaurreproductionandtheirevolutionaryand

ecologicalimplications:Linkingfossilevidencetoallometriesofextantcloserelatives.PLOSONE.

2013;8(8):e72862.https://doi.org/10.1371/journal.pone.0072862PMID:23991160

22.BanosJH,PepinME,VanWagonerN.Class-wideaccesstoacommercialStep1questionbankduring

preclinicalorgan-basedmodules:apilotproject.AcadMed.2018;93(3):486-490.https://doi.org/10.

1097/ACM.0000000000001861PMID:28817433

InstitutionalUSMLEperformancedifferencesPLOSONE|https://doi.org/10.1371/journal.pone.0224675November4,20199/10

23.DanielM,FlemingA,GrochowskiC,HarnikV,KlimstraS,MorrisonG,etal.Whynotwait?Eightinstitu-

tionssharetheirexperiencesmovingUnitedStatesMedicalLicensingExaminationStep1aftercore clinicalclerkships.AcadMed.2017;92(11):1515-1524.https://doi.org/10.1097/ACM.

0000000000001714PMID:28422816

24.NationalResidentMatchingProgram,DataReleaseandResearchCommittee:Resultsofthe2018

NRMPProgramDirectorSurvey.NationalResidentMatchingProgram,Washington(DC):2018.Avail- ablefrom:https://www.nrmp.org/wp-content/uploads/2018/07/NRMP-2018-Program-Director-Survey- for-WWW.pdf

25.MorseR,MartinE.Methodology:2020BestMedicalSchools.Availablefrom:https://www.usnews.

com/education/best-graduate-schools/articles/medical-schools-methodology

26.ZhaoX,OpplerS,DunleavyD,KroopnickM.Validityoffourapproachesofusingrepeaters'MCAT

scoresinmedicalschooladmissionstopredictUSMLEStep1totalscores.AcadMed.2010;85(10 Suppl):S64-67.https://doi.org/10.1097/ACM.0b013e3181ed38fcPMID:20881707

27.JurichD,DanielM,PaniaguaM,FlemingA,HarnikV,PockA,etal.MovingtheUnitedStatesMedical

LicensingExaminationStep1aftercoreclerkships:Anoutcomesanalysis.AcadMed.2019;94(3):

371-377.https://doi.org/10.1097/ACM.0000000000002458PMID:30211755

InstitutionalUSMLEperformancedifferencesPLOSONE|https://doi.org/10.1371/journal.pone.0224675November4,201910/10