[PDF] Parameter-Free Lens Distortion Calibration of Central Cameras

View - Download Parameter-Free Lens Distortion Calibration of Central Cameras

Previous PDF

Next PDF

FilippoBerg amasco,LucaCosmo,AndreaGasparetto,AndreaAlbarelliandAndrea Torsello DipartimentodiScienze Ambientali,Informaticae Statistica

Universit

`aCa"F oscari-V enice,Italy

Abstract

Atthecor eofmany ComputerVisionapplicationsstands

theneedto defineamathematical modeldescribingthe imagingprocess.To thisend,thepinholemodelwith ra- dialdistortionis probablythe mostcommonlyused, as itbalanceslow complexitywith aprecision thatissuffi- cientformost applications.Onthe otherhand,uncon- strainednon-parametricmodels,despite beingoriginally proposedtohandlespecialtycamer as,havebeen shown tooutperformthe pinholemodel,e venwiththe simplerse- tups.Still,notwithstanding thehigheraccur acy,the inabil- ityofdescribing theimaging modelbysimple linearpro- jectiveoperator sseverelylimitstheuseofstandar dalgo- rithmswithunconstr ainedmodels.In thispaperwepropose aparameter -freecameramodelwhereeachimagingr ayis constrainedtoacommonoptical center,for cingthecam- eratobecentral. Suchmodel canbeeasily calibratedwith apractical procedurewhichpr ovidesaconvenientundis- tortionmapthat canbeused toobt ainavirtual pinhole camera.Theproposedmethod canalsobe usedtocalibrate astereo rigwithadisplacementmapthat simultaneously providesstereorectificationandcorr ectslensdistortion.

1.Introduction

Thepinholecamera modelisby farthe mostwidelyused

imageformationmodel. Thereasonsbehind itshugesuc- cessamongresearchers andpractitionersis duetose veral factors.First,itssimplicity. Infact, themodelis fullyde- terminedbyits opticalcenterand principalpoint.Gi ven thisbasicinformation, thewholeimaging processcanbe modeledasa directprojectionof thesceneonto theimage plane.Second,the availability ofmathematicalt ools.Its plainformulationallo wstoeasily applyawidespectrum ofpowerful andwellunderstoodmathematicaltools,rang- ingfromepipolar geometryto projective invarianceofcon- ics[6]and straightlines[10]. Finally,its widerangeof applicability.Specifically,provided thatthefieldofview ofthecamera fullyintersectsthe imageplane,a properlens distortionmodelcan beappliedto approximateanideal pin-

hole.Generallyspeaking, theroleof thedistortionmodel istodescribe howthe incominglightrays divergefromthe expectedcentraldirectiononcethe yenterthe lensassemblyofareal camera.Inpractice, sinceforan ygiv enlenscon- figuration,thegeometry thatrelatesthe opticstothe imagesensorisfix ed,thedistortion modelisoftendefinedjustas adisplacement functiontobeappliedov ertheimage plane.Moreover,duetophysicalconsiderations,manydistortion modelsareradial; thatis,the yaredefined asadisplacement functionthatonly dependsonthe distancefromthe princi-palpoint.This isthecase withtheseminal modelproposedbyTsai[29], wherethedistortion functionismodeled asapolynomialradialtranslation withtwo non-zerocoefficients respectivelyforthesecondandfourthde greeterms.Such model,albeitsimpler thanprevious proposals[7,8], ob-tainedalar gesuccess,mainly becauseitwasableto offeragoodapproximation oftheimaging processandan easycalibrationprocedurew asav ailablesinceitsintroduction.Forthisreason,ithas beenembracedand extendedse v-eraltimesin thelastthree decades.Zhang,when intro-ducinghiswell-kno wncalibrationprocedure [31],adoptsthesamemodel withslightmodifications. Morerecently, ClausandFitgibbon [9]proposeda rationalfunctionas areplacementforthe originalpolynomialterm. Thislatterapproachiscurrently oneofthe mostsuccessful,probably becauseofits inclusioninthe OpenCVlibrary. Otherre-centapproachesinclude variationson thenumberand typeofparameters[18, 30,11],the enforcementofprojecti veinvariantsforparameterestimation[10,5,2], simultaneouscalibrationofmultiple cameras[13,12], andextensions de-signedtow orkwithhighly distortedcameras[24,26].In-deed,itis whenaddressingthe needtocalibrate fisheyeand catadioptriccamerasthat theneedto diver gefromthe pin-holemodelis feltthemost, asany pinhole-basedmodel,regardlessofitslevelof sophistication,isgeometrically unabletoproperly describecamerase xhibitingavie wan-glewiderthan 180degrees. Too vercomesuch limitation,severalalternativeparametricmodelsha vebeenproposed.Someofthem trytomodify thecapturedimage inordertofollow theoriginalpinholebehaviour[16]. Othersgotroughatotally differentpath byintroducingno velimage

3847

formationmodels[17, 27].Alsocatadioptric [1,19]and light-field[3]systems have beenwidelyco veredinthelit-erature,witha largeselection ofcalibrationmethods. Giventhehighv ariabilityofnon-pinhole cameras,ithasbeenproventobeverydifficult todefinea parametricdistor- tionmodelable toaccommodatethe diverse behaviourof physicallenses.Thishindranceshas beenaddressedby in-troducinggeneralcamera modelsbasedon unconstrainedrays(usuallycalled raxels)[14, 25]aswell aspartiallynon-parametricdistortionmodels suchas[28] and[15].F orformerapproach,a remarkablerecentcontrib utionisgi venin[20],b utthecalibration islimitedtoasmallset of3Draysthatha veto beinterpolatedtocompriseallthepixelsintheimage space.Thenon-parametric distortionapproachof[15]is similarinspirit toourw orkbut itisnot directlycomparablesincethe distortionisstill modeledasa radialfunctiondependingon thedistancefrom apointon theim-ageplane.Similarlyto[23],we alsoadoptastructured-lighttargetstrategytoobtainadensenon-parametricimageplaneundistortion.Howe ver,[23]onlyusesasingleshotofthecalibrationtarget soitcannotincreasetheaccurac ybya ver- agingsev eraltargetviewsnorbeusedto calibratemultiplecamerasatonce.

modelsandrelated calibrationproceduresa lastresortto adoptonlywhen traditionalapproachesf ailforgeometrical ormethodologicalissues. Thisisdue tothef actthattheir flexibilitycomesattheprice ofahigher calibrationcom- plexityand(sometimes)cruderapproximations. Recentre- searchshows thatafullyunconstrainedimagingmodel can beappliedef fectively toreal-worldpinhole[4]camerasob- tainingbetteraccurac yandwithout needingacomplexcal- ibrationprocedure.Ho wever ,theadvantagesintermsof precisionarepartially offsetby theinabilityto usethewide mathematicaltoolsetde visedforthe pinholemodel.Thisis unfortunate,sincewhen dealingwithstandard cameraswith nolarge distortions,thecentralmodelisstill reasonable. Withthisstudy,we trytofill thegapbetweentraditional pinholecalibrationtechniques andunconstrainedmodels.

Namely,weproposeamodel wheretheonly raxelscon-

straintitto crossacommon projectioncenter. Underthis assumption,afterperforming apropercalibration, itiseasy todefinea non-parametricdisplacementmap over theim- ageplanethat canbeused tomov ebackand forthfromthe unconstrainedmodelto avirtualpinhole camera.This,in turn,allows toexploitthefullarsenal oftoolsdesigned to workwithpinholecameras. Tothis end,thecontrib utionof thispaperis threefold.First,we introduceanef fective cali- brationprocedurefor theproposedsemi-constrained model. Second,wedefine anoptimalapproach tocreatea virtual howtonaturallyextend themethod tothecalibration and rectificationofa stereopair. 2.Nov eltyandApplicationScenario

Differentlyfrommanyother non-parametricmethods,

ourgoalis nottocope withnon-conv entionalimagingge- ometries.Rather, weareindeeddealingwithcentral cam- eras,thatis, imagingdevices whereraysare supposedto convergetoacommonopticalcenter.Thisis adomainthat hasbeentraditionally addressedbypinhole modelsaug- mentedwitha radialdistortionfunction. Thesemodels,and therelatedcalibration methods,areso widelyusedthat it isquitenatural towonder ifitactually makesan ysenseto adoptmorecomple xsolutions.This questionispartially answeredbythe continuedinterestin enhancedcentraldis- tortionmodels[22], whichisa statementofthe active state oftheresearch. Anadditionalhint totheopen natureofthis problem,isalso offeredby recentresearchef fortstoward theadoptionof generaldistortionmodels [21],thatin some casescanbe estimatedev enwithno directrecov erycamera poseandpi nhole parameters[23].Withrespecttothislat- termethod,the generalgoalcould seemvery similartothe techniqueweare introducing.Inf acttheauthors proposean image-basedundistortionestimation thatcanbe performed onasingle imagewithoutthe needforan explicitassess- mentofi nstrinsicande xtrinsiccameramatrices.Howev er, wewould liketostressani mportantaspect:image-based methodsarenot actuallydev oidofprojecti veparameters astheestimated distortionisaf fectedbyan homography H,mixinginstrinsic andextrinsic parameters.Actually, the methodproposedin [23]obtainsan estimateforH assum- ingthedistortion tobezero aroundtheimage center,which isindeeda quitestrongassumption. Unfortunately,this makesitdifficultto effectiv elycombinemorethanonetar - getview .Thisisthereasonbecauseonlyone poseisused. Conversely,weexplicitlyseparatethepose-invariant cam- eraraysfrom thepose-dependenttar getorientation,thus offeringageometricallysoundconstraint amongmultiple exposures.Suchconstraintsbecomee ventighter whensi- multaneouslyaccountingfor multiplecameras,which isen- abledbydesign withourmethod. Tothis end,ourapproach iswellsuited inallthe scenarioswherethe highaccuracy of generalmodelsis sought,stillthe notionofan imageplane isneeded.This isthecase formostsetups whereepipolar geometryise xploited,including(for instance)structured- lightscanning,stereo reconstructionandmultiple camera tracking.Itshould bealsonoted thatthesekind ofappli- cationsareactually theonesthat benefitthemost fromthe abilityofour methodtoeasily computeanunified rectifica- tionandundistortion mapthatcan spanmultiplecameras. Finally,theexperimentalassessment oftheactual impact ofthemodel isalsoan usefulcontribution ofthispaper . Specifically,weshowthat ourhybrid approachyieldsan enhancedperformancewith respecttothe parametricdistor- tionmodel,while retainingtheaforementioned advantages relatedtothe availability ofanactual imageplane. 3848

Figure1.Schema oftheoptimized cameramodelin volving theopticalcentero,theray direction,theobserv edande xpectedcode

andatar getpose.

3.UnconstrainedDistortion Calibration

Toestimateadensenon-parametric lensdistortionwe

startbyreco veringthe 3Dlightraysassociatedtoeachim- agepixel. Specifically,weformalizethelight pathenter- ingthelens andhittingthe sensoratdiscrete pixelcoordi- nate(u,v)withthestraight liner(u,v)=(o,d(u,v))passing troughthepoint oandorientedalong thedirectiond(u,v).

Thecommonpoint oconstrainsourmodel tobecentral

whilenorestriction isenforcedon thedirectionsd(u,v). Also,theuniform-spaced gridofthe CCDprovides anor- deringonthe raysspatialtopology .

Sincethemodel implies3de greesoffreedom foreach

pixel,plusadditional3for theopticalcenter o,standard point-to-pointcalibrationapproaches like[31, 29,9]cannot provideenoughdatatoreco vera validsolution. Wesolv e thisbyadopting thedensestructured-light targetproposed in[4].Specifically ,weuse anhigh-resolutionLCDscreen displayingphaseshift patternstouniquely localizeeachtar - getpointseen bythecamera.

Thishasa strongadvantage withrespectto commoncal-

ibrationtargets composedbyadiscretesetof features:un- likemethodsbasedoncorner orellipseloca lization,wecan reasoninterms ofdiscretecamera coordinates.Indeed,for eachcamerapix el(u,v)aprecise sub-pixellocalizationof the2Dtar get-spacecoordinateof theobservedpointcanbe recoveredfromthephaseunwrappingprocess.In addition, thecodingtheory approachisrob ustagainst possiblevisual artifactsthatmayappearon targetsurf acelike illumination gradients,specularhighlights andshadows.Finally,wetake playstoget anaffordable targetf armoreaccurate thatthe onescreatedthrough inkprintingprocess.

Toestimatetheopticalcenter andthedirection ofeach

ray,thecalibrationtarget isex posedto thecameraindif- ferentpositionsand orientations.We denotewithRTsthe

3×4matrixdescribingtheroto-translationoft hetargetwith

respecttothe camerainthe poses,assumingthe targetref-

erenceframelocated intheupper leftmonitorpix elwith?i,?j,?kversorsbeingrespectively themonitorcolumns, rowsandnormal.F oreachpose, letCos(u,v)?R2bethecode

observedbytherayr(u,v)inshots.

SincetheLCD geometryandthe displayedcodesare

known,wecantake advantageof aposeRTstomapeach codetoa 3Dpointin thecameraspace, viceversa. Forin- stance,theintersection betweenaray r(u,v)andthetar get planedefinedby aposeRTsyieldstothe expectedcode

Ce(r(u,v),RTs)?R2thattheray shouldhav eobserved

(Fig.1).

3.1.SingleCamera calibration

Following[4],werecoverthe geometryofthe raysen-

tionproblem: argmin r (u,v),RTs? alsonthe targetplane betweentheobserv edandexpected codesand(Σs(u,v))-1istheerror covariance matrixforthe givenray-posecombinationthataccountsforerrors het- eroscedasticityinthe imageplane.

Inoursetting, weaimto simultaneouslyminimizethe

opticalcentero,thedirection d(u,v)ofallrays andthe pose RT sforeache xposureofthe target.Similarlyto[4],we canalsotak eadvantage oftheconditionalindependence oftheparameters toimplementan alternatingoptimization schemethatseeks optimaloandd(u,v)assuminglastesti- mationofRTsfixed,andvice-versa.While ouroptimiza- tioninv olveslessparameters,theoptimizationitselfismore complexsincethecommonoptical centerintroducesa cou- plingbetweenthe rayswhichcannot beestimatedindepen- dentlyanymore. Asaconsequence,theraysoptimization stepsimultaneouslyestimates theopticalcenter oandthe raydirectionsd(u,v)givenalltheposes.Forthe poseesti- mationstepwe adoptthesame ICP-basedoptimizationin- troducedin[4]. Theformerstep isdiscussedin detailin section3.1.1while, forthelatter ,werefer thereaderto the originalpaper. Tostartthealternatingoptimiza tionweneed agoodini- tialapproximationfor theinv olvedparameters. Tothis end, wegather asetof3D-2Dpointcorrespondences assum- ingadiscrete gridoftar getpointssimilar towhatcan be commonlyobtainedwith achessboard.Then, weusecali- brateCamerafunctionprovided byOpenCVtoobtaintarget posesforeach exposureand thedirectionof eachray.Note thatstartingcondition isnota criticalaspectsince most baselinecalibrationmethods aremorethan adequatetopro- videareliable initialconfiguration,especially whendealing withcamerasthat canbeassumed tobealmost central(dif- ferently,themethodproposedin [4]shouldbe preferred). Inaddition,the iterative methodisbased onaleast-square minimizationwhichis guaranteedtocon verge toa(possibly local)minimum. 3849

3.1.1OpticalCenter andRaysDir ectionOptimizationIntheoandd(u,v)optimizationstepwe considertarget

posesconstant.Let x s (u,v)=RTs( (Cos(u,v) 0 1) bethe3D coordinatesof theobserved codeCos(u,v)transformedtroughthe poseRTs.Assho wnin[4], thegen- eralizedleastsquares formulationwithrespect tothetar get coordinatescorrespondsto alinearleast squareswiththe distanceofeach rayandits associated3Dpoint xs(u,v).We canformulatethe estimationofthe opticalcenteroas: argmin o? u,vmind (u,v)? s?(hs(u,v))T(I-d(u,v)dT(u,v))?2(2) wherehs(u,v)=(xs(u,v)-o),andthe internalminimiza- tionaptto findthebest d(u,v)minimizingthesum of squareddistancesbet weenr(u,v)andallthe xs(u,v).We startbyre-writing thesquarednorm in(2)as (hs(u,v))T(I- d (u,v)dT(u,v))hs(u,v)toobtain argmin o? u,v? s?hs(u,v)?2-maxd (u,v)? s? dT(u,v)hs(u,v)? 2(3) Let¯x(u,v)bethecentroid ofthepoint cloudgeneratedby theintersectionsof therayr(u,v)andthetar getforeach ob- servedpose.Also,let¯h(u,v)=(¯ x(u,v)-o)bethedistance vectorbetweenowithsuchcentroid. Byexpressing hs(u,v)asthesummation ofthetw ocomponents: h s (u,v)=(xs(u,v)-¯x(u,v))+¯h(u,v) andexpanding theformulationin(3)weobtain: argmin o? u,vN (u,v)?tr(S(u,v))+?¯h(u,v)?2?-(4) -maxd (u,v)dT(u,v)? N (u,v)S(u,v)+N(u,v)¯h(u,v)¯hT(u,v)? d (u,v)(5) whereS(u,v)andN(u,v)arerespectiv elythecovariancema- trixandthe sizeofthe pointcloudgenerated byr(u,v).

Sincewest artouroptimization withaconfiguration

closetothe optimum,wee xpectthatthe distancebetween eachrayand itsexpected codeisas smallasfe wtargetpix- els.Thisimplies thatthespatial extentof eachpointcloud isorderof magnitudesmallerthan thedistance?¯h(u,v)?2. Underthisassumption, anapproximatemaximizer for(5) isgiv enby d (u,v)=¯h(u,v) ?¯h(u,v)?2(6) Bysubstituting(6) into(4)and (5),aftersome simplifi- cations,weobtain thefollowing alternative formulation

CeuCev

10241280 00

Figure2.Left: RMSebetweenobserv edande xpectedcodesfor andcalibrationtar getposesreco veredbytheoptimization.Only a subsetofthe cameraraysare plottedforvisualization purposes. argmax o? u,vN (u,v)¯ hT(u,v)S(u,v)¯h(u,v) ?¯h(u,v)?2(7) Problem(7)cannot besolved inaclosed form.To providea goodapproximatesolution wecomputethe derivati vewith respecttoo: ∂o? u,vN(u,v)¯ hT(u,v)S(u,v)¯h(u,v)?¯h(u,v)?2(8) u,v2N(u,v)K(u,v)¯h(u,v) K (u,v)=? ?¯h(u,v)?4(9)

IfK(u,v)isknown, ocanbeobtained bysettingto zero

Equation(8)and solvingtheresulting linearsystem:

u,v2N(u,v)K(u,v)o=? u,v2N(u,v)K(u,v)¯x(u,v)(10) SinceK(u,v)isitselfafunctionofo,themaximization prob- lem(7)is tacklediterativ elybycomputing K(u,v)withthe estimateofoatiterationt-1andthensolving (10)toob- tainane westimateat iterationtandrepeatingthis process until?o(t)-o(t-1)?3.1.2From raybundletovirtualpinhole camera Aftertheoptimization ofrays,optical centerandposes we obtainadetailed modeldescribingthe lightpathentering thecamera.Ne xt,weneed tochooseaconvenient image planethatdefine theintrinsicparameters ofane wvirtualquotesdbs_dbs8.pdfusesText_14

[PDF] opencv distortion model

[PDF] opencv radial and tangential distortion

[PDF] opencv python tutorials documentation pdf

[PDF] opening business account

[PDF] openldap 2.4 setup

[PDF] openldap administrator's guide

[PDF] openldap create database

[PDF] openldap lib

[PDF] openldap mdb

[PDF] openldap sdk

[PDF] operant conditioning

[PDF] operating modes of 8086 microprocessor

[PDF] operation research question bank with answers pdf

[PDF] operation research questions and answers pdf

[PDF] operational process of state prisons