[PDF] Structure-from-Motion Revisited - Johannes L. Schönberger12

View - Download Structure-from-Motion Revisited - Johannes L. Schönberger12

Previous PDF

Next PDF

Structure-from-MotionRevisited

JohannesL.Sch

¨onberger1,2?,Jan-MichaelFrahm 1

1UniversityofNorthCarolinaatChapelHill

2Eidgen¨ossischeTechnische HochschuleZ¨urich

jsch@inf.ethz.ch,jmf@cs.unc.edu

Abstract

IncrementalStructure-from-Motionis aprevalentstrat- egyfor3Dreconstruction fromunor deredima gecollec- tions.Whileincr ementalreconstruction systemshave tremendouslyadvancedinallr egards, robustness, accu- racy,completeness,andscalabilityremainthe key problems towardsbuildingatruly general-purposepipeline.W epro- poseane wSfMtec hniquethatimprovesupon thestateof theartto makea furthersteptow ardsthisultimategoal. Thefullr econstructionpipelineis releasedtothepublicas anopen-source implementation.

1.Introduction

Structure-from-Motion(SfM)from unorderedimages

hasseentremendous evolution over theyears.Theearly self-calibratingmetricreconstruction systems[

42,6,19,

16,46]served asthefoundationforthefirst systemson

unorderedInternetphoto collections[

47,53]andurban

scenes[

45].Inspired bytheseworks,increasinglylar ge-

scalereconstructionsystems have beendev elopedforhun- dredsofthousands [

1]andmillions [20,62,51,50]tore-

centlyahundred millionInternetphotos [

30].Av ariety

ofSfMstrate gieshav ebeenproposedincludingincremen- tal[

53,1,20,62],hierarchical[ 23],andglobal approaches

14,61,56].Arguably ,incrementalSfMisthemostpopular

strategyforreconstructionofunordered photocollections. Despiteitswidespread use,westill have notaccomplished todesigna trulygeneral-purposeSfM system.Whilethe existingsystemshav eadvancedthestateof thearttremen- dously,robustness,accuracy ,completeness,andscalability remainthek eyproblems inincrementalSfMthatpreventits useasa general-purposemethod.In thispaper, wepropose anew SfMalgorithmtoapproachthisultimate goal.The andthecode iscontributed totheresearch communityasan open-sourceimplementationnamed COLMAPavailableat https://github.com/colmap/colmap. ?Thiswork wasdoneattheUni versityofNorthCarolinaat ChapelHill. Figure1.Result ofRomewith 21Kregistered outof75K images.

2.Review ofStructure-from-Motion

SfMisthe processofreconstructing 3Dstructurefrom

itsprojectionsinto aseriesof imagestaken fromdifferent viewpoints.IncrementalSfM(denotedas SfMinthispaper) isasequential processingpipelinewith aniterativ erecon- structioncomponent(Fi g.

2).Itcommonly startswithfea-

tureextraction andmatching,followedbygeometric verifi- cation.Theresulting scenegraphserv esasthe foundation forthereconstruction stage,whichseeds themodelwith acarefullyselected two-view reconstruction,beforeincre- filteringoutliers,and refiningthereconstruction usingbun- dleadjustment(B A).Thefollo wingsectionselaborateon thisprocess,define thenotationused throughoutthepaper , andintroducerelated work.

2.1.Correspondence Search

Thefirststage iscorrespondencesearch whichfinds

sceneov erlapintheinputimagesI={Ii|i=1...NI} andidentifiesprojections ofthesame pointsino verlapping images.Theoutput isaset ofgeometrically verifiedimage pairs¯Candagraph ofimageprojections foreachpoint.

FeatureExtraction.ForeachimageIi,SfMdetects sets

F i={(xj,fj)|j=1...NFi}oflocalfeatures atloca- tionxj?R2representedbyan appearancedescriptorfj. Thefeaturesshould beinv ariantunderradiomet ricandge- ometricchangesso thatSfMcan uniquelyrecognizethem inmultipleimages [

41].SIFT[ 39],itsderi vativ es[59],and

morerecently learnedfeatures[

9]arethe goldstandardin

termsofrob ustness.Alternativ ely,binaryfeaturesprovide betterefficienc yatthecostofreducedrobustness[ 29].
4104
Correspondence SearchIncremental ReconstructionImagesReconstruction

Initialization

Bundle AdjustmentTriangulation

Feature Extraction

Matching

Geometric Verification

Image RegistrationOutlier Filtering

Figure2.Incremental Structure-from-Motionpipeline.

Matching.Next,SfMdiscovers imagesthatsee thesame

scenepartby leveraging thefeaturesFiasanappear ance descriptionofthe images.Thena

¨ıveapproachtestsev ery

imagepairfor sceneov erlap;itsearches forfeaturecor - respondencesbyfinding themostsimilar featureinimage I aforev eryfeatureinimageIb,usinga similaritymet- riccomparingthe appearancefjofthefeatures. Thisap- proachhascomputational complexityO(N2IN2F i)andis prohibitiveforlargeimagecollections.A varietyof ap- proachestacklethe problemofscalable andeffici entmatch- ing[

1,20,37,62,28,49,30].Theoutput isaset ofpoten-

tiallyov erlappingimagepairsC={{Ia,Ib}|Ia,Ib?

I,a< b}andtheirassociated featurecorrespondences

M ab?Fa×Fb. GeometricVerification. Thethirdst ageverifies thepo- tentiallyov erlappingimagepairsC.Sincematching is basedsolelyon appearance,itis notguaranteedthat cor- respondingfeaturesactually maptothe samescenepoint. Therefore,SfMv erifiesthematches bytryingtoestimatea transformationthatmaps featurepointsbetween imagesus- ingprojectiv egeometry.Dependingonthespatialconfig- urationofan imagepair, differentmappings describetheir geometricrelation.A homographyHdescribesthetrans- formationofa purelyrotatingor amoving cameracapturing aplanarscene [

26].Epipolargeometry [26]describesthe

relationfora movingcamera throughtheessential matrix E(calibrated)orthe fundamentalmatrixF(uncalibrated), andcanbe extendedto threeviews usingthetrifocalten- sor[

26].Ifa validtransformation mapsasuf ficientnumber

offeaturesbetween theimages,the yareconsidered geo- metricallyverified. Sincethecorrespondencesfrommatch- ingareoften outlier-contaminated,rob ustestimationtech- niques,suchas RANSAC[

18],arerequired. Theoutput

ofthisstage isaset ofgeometrically verifiedimage pairs¯C, adescriptionof theirgeometricrelation Gab.To decideon theappropriaterelation, decisioncriterions likeGRIC [ 57]
ormethodslik eQDEGSAC [

21]canbe used.Theoutput

ofthisstage isaso-cal ledscenegraph [

54,37,48,30]with

imagesasnodes andverified pairsofimages asedges.

2.2.Incremental Reconstruction

Theinputto thereconstructionstage isthescene graph.

Theoutputsare poseestimatesP={Pc?SE(3)|c=

1...NP}forregistered imagesandthereconstructedscene

structureasa setofpoints X={Xk?R3|k=1...NX}. Initialization.SfMinitializesthe modelwitha carefullyselectedtwo-vie wreconstruction[

7,52].Choosinga suit-

ableinitialpair iscritical,since thereconstructionmay neverrecoverfromabad initialization.Moreover,thero- bustness,accuracy,and performanceofthereconstruction dependsonthe seedlocationof theincrementalprocess. Initializingfroma denselocationin theimagegraph with manyoverlappingcameras typicallyresultsinamorerobust andaccuratereconstruction duetoincreased redundancy. Inquotesdbs_dbs3.pdfusesText_6

[PDF] structure of supreme court

[PDF] student dependent visa france

[PDF] student film competitions 2020

[PDF] student ikamet

[PDF] studio 1 qu'est ce que tu fais

[PDF] study guide for linux essentials exam

[PDF] studytee font

[PDF] sub saharan africa population 2019

[PDF] subfields of sociology

[PDF] subjective well being pdf

[PDF] successful event planning pdf

[PDF] sucrose cryopreservation

[PDF] suddenlink

[PDF] sugar cane molasses specification

[PDF] sugar free bakery columbus ohio