[PDF] DéjàVu: A Map of Code Duplicates on GitHub

View - Download DéjàVu: A Map of Code Duplicates on GitHub

Previous PDF

Next PDF

84

DéjàVu:AMapofCodeDuplicates onGitHub

CRISTINAV. LOPES,UniversityofCalifornia,Irvine ,USA

PETRMAJ,CzechTechnicalUniv ersity,CzechRepublic

PEDROMARTINS,UniversityofCalifornia,Irvine ,USA

VAIBHAVSAINI,UniversityofCalifornia,Irvine ,USA

DIYANG, UniversityofCalifornia,Irvine ,USA

JAKUBZITNY,CzechTechnicalUniv ersity,CzechRepublic

HITESHSAJNANI,MicrosoftResearch,USA

JANVITEK, NortheasternUniv ersity,USA

Previousstudieshaveshownthat thereis anon-trivialamountofduplicationin sourcecode. Thispaper analyzesacorpus of4.5million non-forkproje ctshosted onGitHubrepresentingover428million !leswritten inJava,C++, Python,andJavaScript. Wefound thatthiscorpus hasamer e85 millionunique les.Inother words,70%ofthecode onGitHubconsistsofclones ofpre viouslycreate d!les.There isconsiderablevariation betweenlanguageecosystems.JavaScripthasthehighestrate of leduplication,only 6%ofthe lesar edistinct. Java,onthe otherhand,has theleastduplication, 60%of lesar edistinct.Lastly,aproject-lev elanalysis shows thatbetw een9%and31%oftheprojectscontainat least80%of lesthatcan befound elsewhere .Theserates

ofduplicationhav eimplicationsfor systemsbuiltonopensour cesoftwar easw ellasfor researchers interested

inanalyzinglarge codebases. Asa concreteartifactofthisstudy ,w ehavecreated DéjàVu,apublicly available

mapofco deduplicatesin GitHubrepositories. CCSConcepts:•Informationsystems"Near-duplicateandplagiarism detection;•Softwareandits engineering"Ultra-large-scalesystems; AdditionalKeyW ordsandPhrases:CloneDetection,Sour ceCodeAnalysis

ACMReferenceFormat:

CristinaV. Lopes,PetrMaj,Pe droMartins,VaibhavSaini, DiYang, JakubZitny,Hitesh Sajnani,andJan Vitek.2017.Déjà Vu: AMapofCodeDuplicatesonGitHub. Proc.ACMProgram.Lang.1,OOPSLA, Article84

1INTRODUCTION

Theadvent ofweb-hostedopen sourcer epositoryservicessuchasGitHub,BitBucketandSource- Forgehavetransformedhow sourcecodeisshar ed.Creatingapr ojecttakesalmostno e#ortandis freeofcostforsmallteams working intheop en.Ov erthelast twodecades,millionsof projects havebeenshar ed,buildingupamassivetro veoffreesoftware .Anumb erofthese projects have beenwidelyadoptedandar epartof ourdailysoftwareinfrastructure .More recentlythere have beenattemptstotreattheop ensource ecosystemas amassivedatasetand tomineit inthehopes of ndingpatternsof interest.

Authors'addresses:CristinaV .Lopes,Universityof California,Irvine ,USA;PetrMaj,Cze chTechnicalUniversity ,Czech

Republic;Pe droMartins,UniversityofCalifornia,Irvine ,USA;V aibhavSaini,UniversityofCalifornia, Irvine, USA;Di Yang,

UniversityofCalifornia,Irvine ,USA;Jakub Zitny,Czech TechnicalUniversity,Czech Republic;HiteshSajnani, Microsoft

Research,USA;JanVitek,Northeastern University ,USA.

Permissiontomake digitalor hardcopies ofpartor allofthis workforpersonalor classroom useis grantedwithout fee

providedthatcopiesarenotmadeor distributedfor pro torcommer cialadvantageand thatcopiesbearthisnotice and thefull citationonthe rstpage. Copyrightsforthird-partycomp onentsofthisworkmust be honored.Forallother uses, contacttheo wner/author(s).

©2017Cop yrightheld bytheowner/author(s).

2475-1421/2017/10-ART84

https://doi.org/10.1145/3133908 Proc.ACMProgram.Lang.,V ol.1,No.OOPSLA,Article 84.Publicationdate:October2017. Thiswork islicensedunderaCreati veCommons Attribution4.0 InternationalLicense.

84:2C.Lop es,P. Maj,P.Martins,V. Saini,D.Yang,J. Zitny,H. Sajnani,andJ.Vitek

Whenworking withsoftware,onemay wanttomake statementsaboutapplicabilityof,say, acompileroptimization ora staticbug ndingtechnique .Intuitively,onewouldexp ectthata conclusionbased onasoftwarecorpusmade upofthousands ofprograms randomlyextractedfr om anInternet archive ismorelikelytoholdthanonebase dona handfulofhand-pickedbenchmarks suchas[ Blackburnetal .2006]or[ SPEC1998].Foran example, consider[Richardsetal.2011] whichdemonstrate dthatthedesignofthe Mozillaoptimizingcompiler wasskew edby thelackof representativebenchmarks.Lookingatsmallworkloads gaveavery di erentpicturefrom what couldbe gleanedbydownloading thousandsofwebsites. Scalingtolarge datasetshasits challenges.Whereas smalldatasetscan be curatedwith care, largerco debasesareoftenobtaine dbyrandom selection.IfGitHubhasov er4.5millionprojects, howdoesonepick athousandprojects?If statisticalreasoning istob eapplied, thepr ojectsmust beindependent.Independence ofobservationsistakenfor grantedinmanysettings, butwith 7 7 7 7 7 7 7 7 7 7 6 10 7 9 7 3 12 3 7 22
0 10 10 10 9 10 10 10 10 10 10 10 9 9 11 10 8 11 4 18 23
30
0 0 11 10 10 11 10 10 11 11 11 10 11 10 10 11 9 12 11 7 9 7 18 17 100
11 11 12 11 11 11 11 11 11 12 11 12 13 10 10 13 8 15 11 18 12 5 13 13 13 13 13 13 13 13 13 13 14 13 12 10 11 14 14 18 13 8 2 52
15 15 15 15 15 15 15 15 15 14 15 15 15 18 16 12 15 19 13 22
43
0 86
18 18 18 18 18 18 17 17 18 19 18 20 19 19 16 17 20 30
24
20 2 22
22
22
22
22
21
22
23
23
21
22
23
21
22
27
23
19 17 24
9 3 30
30
30
30
30
30
31
29
31
30
31
28
28
33
29
27
25
40
49
0 47
35
37
35
36
36
36
35
36
36
35
37
36
30
48
45
47
36
30
60
55
31
0 39
39
40
40
39
41
40
43
40
40
45
44
44
43
37
35
30
52
63
26
84
46
49
49
48
46
49
49
51
48
46
46
44
43
48
70
73
48
13 34
58
54
53
53
53
55
55
54
53
54
53
53
56
47
46
51
46
14 100
99
0 59
62
57
57
59
59
59
56
56
56
60
58
39
63
35
quotesdbs_dbs22.pdfusesText_28

[PDF] angular project structure example github

[PDF] angular projects for practice

[PDF] angular projects with source code free download

[PDF] angular routing tutorial step by step

[PDF] angular sample project git

[PDF] angular sample project step by step

[PDF] angular sample projects for beginners

[PDF] angular service interface naming convention

[PDF] angular step by step for beginners

[PDF] angular tutorial for beginners 2020

[PDF] angular tutorial for beginners learn angular from scratch

[PDF] angular tutorial for beginners learn angular from scratch mosh

[PDF] angular tutorial for beginners step by step

[PDF] angular tutorial for beginners step by step javatpoint

[PDF] angular tutorial for beginners step by step youtube