[PDF] [PDF] Automated 3D Face Reconstruction From Multiple Images - AMiner

[PDF] Deep 3D Portrait From a Single Image - CVF Open Access

A two-step geometry learning scheme is proposed to learn 3D head reconstruction from in-the-wild face images, where we first learn face shape on single images 

[PDF] 3D Face Reconstruction from a Single Image using a Single

Index Terms—Computer vision, photometry, shape from shading, 3D reconstruction, lighting, single images, face, depth reconstruc- tion 3 1 INTRODUCTION

[PDF] Automatic 3D Face Reconstruction from Single Images - Uni Siegen

An automated algorithm for aligning a 3D face model to a single image has been proposed by Gu and Kanade [14] Based on a Morphable Model of 3D shape 

[PDF] Estimating coloured 3D face models from single images: An

surface texture of a human face from a single image The method draws on a general flexible 3D face model which is “learned” from examples of individual 

[PDF] Regressing a 3D Face Shape From a Single Image - The Computer

To support the ability of our method to reliably reconstruct 3D shapes, we introduce a simple method for head pose es- timation using a single image that reaches 

Deep 3D Portrait From a Single Image - IEEE Xplore

A two-step geometry learning scheme is proposed to learn 3D head reconstruction from in-the-wild face images, where we first learn face shape on single images 

[PDF] Feature-Preserving Detailed 3D Face Reconstruction from a Single

Figure 1: Our single image to 3D reconstruction results compared with related referenced works Robustness and feature-preservation are two major components 

[PDF] Detailed 3D Face Reconstruction from a Single RGB Image

3D Wrinkle Reconstruction, Face Analysis, Optimization 1 INTRODUCTION Reconstructing the 3D human geometry from RGB images has been extensively  

[PDF] Automated 3D Face Reconstruction From Multiple Images - AMiner

We propose a method that reconstructs individual 3D shapes from multiple single images of one person, judges their quality and then combines the best of all 

[PDF] 3D Face Reconstruction from 2D Images - University of Utah School

to assist the reconstruction of the 3D face Thus it makes the area of 3D shape reconstruction from 2D images a complex and a problematic one The topic, 3D 

[PDF] 3d histogram python 3

[PDF] 3d histogram python opencv

[PDF] 3d histogram python pandas

[PDF] 3d image from single photo

[PDF] 3d image generation

[PDF] 3d image histogram python

[PDF] 3d image reconstruction software

[PDF] 3d model background images

[PDF] 3d model from 2d images

[PDF] 3d model from images online

[PDF] 3d model from images open source

[PDF] 3d model from images python

[PDF] 3d model from single image

[PDF] 3d model images hd

[PDF] 3d model reference images

Automated3DFaceReconstruction fromMultiple Images

usingQualityMeasur es

MarcelPiotraschke andVolkerBlanz

InstituteforV isionandGraphics, UniversityofSiegen, Germany

Abstract

Automated3Dreconstructionof facesfrom imagesis

challengingiftheimag ematerialis difficultin termsof pose,lighting,occlusionsand facialexpressions,andif the initial2Dfeatur epositionsar einaccurateorunreliable .We proposeamethodthatr econstructsindividual3D shapes frommultiplesingleimag esofone person,judg estheir qualityandthen combinesthebest ofallr esults.Thisis doneseparately fordifferentre gionsofthe face.Thecore elementofthis algorithmandthe focusofour paperisa qualitymeasure thatjudgesareconstruction withoutinfor- mationaboutthe trueshape. Wee valuatediffer entqual- itymeasures, developamethodforcombining results,and presentacompleteprocessing pipelineforautomated re- construction.

1.Introduction

Algorithmsthatreconstruct 3Dfaces fromimagesby

fittingadeformable facemodel, suchasa 3DMorphable Model(3DMM),rely onarelati velyprecise initialposition- ingofthe face[7] orona setoffeaturepointcoordinates [8].For anautomatedprocedure,itisstraight-f orw ardto detection,suchas thealgorithmby ZhuandRamanan [33] orotherfeature detectors[11].In practice,howe ver, this combinationhasturned outtobe morechallengingthan ex- pected,posinga numberoffundamental questions.The fea- turepointdet ectionisa non-trivialtask,especiallyifthe im- agematerialincludes complexlighting, faciale xpressions, wrinkles,eye glassesorfacialhair. Therefore,thefeatures maybeinaccurate, andsomemay even beoutliers.More- over,theoptimalsetoffeaturesfor3DMMfitting includes pointsthatare noteasyto detect,suchas thefacial silhou- etteandthe ears.Thosepoints arenecessaryfor the3DMM toconv ergetothecorrectposeangle,andthisinturnaf fects theshapeestimate.

Therefore,asimple combinationofe xistingmethods

Figure1:A segment-based,weighted linearcombinationis usedtocreate thefinalhead shape.Theweight decreases withtherank. Implausiblese gmentsarediscarded. Note thateachf acialsegment ishandledseparately.Optionally thetexture canbeextractedfromone oftheinput images. producesresultsthat aresubstantiallyw orsethanthose ob- tainedwithmanually labeledfeatures.Attempts tomake

3DMMfittingmore robust[10] arepromisingb utstillnot

sufficient.Instead,weargue thatinman yreal-world appli- cationsmorethan oneimageof apersonis available, soan automatedalgorithmcan exploitredundant datafrommulti- pleimagesto gainrob ustnessandreliability .Ouralgorithm outperformsexisting methodsofsimultaneous3Drecon- structionfrommultiple images[7]significantly ,whichmay bedueto thefact thatoutliersin featurepositionsadv ersely affectthesimultaneousleastsquares solution. Incontrast,our algorithmcalculatesseparate reconstruc- tionsfromeach inputimage,and thencombinesthem toan optimalov erallsolution.Weproposeamethodthatselects themostplausible reconstructions,operateson differentre- gionofthe faceseparately ,andmer gesthemintoasingle

3Dface.

Theke ycomponentofouralgorithmisanew measure

3418

forthevisual qualityof3D reconstructions,basedon sur-facenormals.Automatedassessmentof visualqualityin computergraphicsand visionisa fundamentalchallenge.Simpleimagecomparisons areinsufficie ntbecausethe yareinsensitivetosmallbutimportant errorsandartif acts.Eu-clideandistancein 3Dov erratesglobalshape deformationsthatwould beirrelevanttohuman observers.Mahalanobis distanceisalso inconsistentwiththe qualityratingsof hu-mans.Inan experimentalcom parisonwithquality ratingsfromhumansubjects, ournew ,normalbased measureout-performsthesee xistingcriteria.

Insummary, thecontributionsofthispaper are:

•ageneralmeasure ofthequality (naturalness)ofa shapereconstruction,

•analgorithmfor selectingandcombining reconstruc-tionsofdif ferentfacial regions(segments)fromdif fer-entinputimages intoasingle 3Dface,

•anautomatedalgorithm thatproduces3D shapere-constructionsfrommul tipleimagesof aperson,whichgoesbeyond asimplecombinationoflandmarkdetec- tionand3DMM fitting.

2.RelatedW ork

Althoughsev eralapproacheshavebeenpublishedre-

latedtohigh quality3Dreconstructions offace sfrom2D images,automatedreconstruction stillremainsa challeng- ingtask,especially withfacial expressions.It isoftendif fi- culttofind imageswitha neutralexpression, asmostpeople tendtosmile inportraits. Intheliterature onface modelingsev eraldifferent ap- proachescanbe found.For highquality3D reconstructions offaces whichareusedincomputer gamesand movies, thestateof thearttechniques stillrequire3D scansofthe personusinglaser scannersormulti-vie wcamerasetups [3,9,4, 5,12, 21,2].Additionally ,substantialpost pro- cessingisrequired tocombinethe generated3Ddata andto morphbetweendif ferentfacial expressionsandvisemesto realisticallyanimatethe subjectsface. Approacheslike theonepresentedinthispaper trytoob- viatetheneed forspecialequipment. Instead,they makeuse ofdatathat canbeeasily producedwithstandard equipment orthatis alreadyav ailable,suchas photoorvideo data.

Multi-viewgeometry[26,14,28, 13]isa commonproce-

duretoreconstruct 3Dshapesfrom several singleimages orvideoframes. Althoughthesealgorithms arequitefle xi- bleinusage fordifferent scenariosvarying fromtherecon- structionofb uildings,smallerobjects andevenfaces, they cannotsufficiently handlenon-rigidtransformations(facial expressions)withinaseriesof inputimages. Otherrecentpublications have shownpromising results byaligninga 3Dface tosingleor multipleimagesas well astovideos frames.Theapproaches byPark et.al[22],

AldrianandSmith [1]andDou et.al[24] reconstructthe3Dshapefrom asingleimage. Wanget al.[30]e xtractthesilhouettefrom several inputimagesto reconstructthe3Dshape,while Rothet.al [27]usean imagecollectionforphotometricstereo-based normalestimationwhich iter-ativelyoptimizesthesurfacereconstruction.By estimatingtheposeand computingtheoptical flow, ahighdetail re-finementofthe 3Dshapeis performed,resultingin a3Dto2Dcorrespondence [20,18,15, 29].Suwajanak ornet.al[29]e vencaptured finedetailslikewrinklesandin[19]Kemelmacher-ShlizermanandSeitzshowedthatalso "facesinthewild" canbehandled properly.But theseapproacheslackanadditional 3Dto3D correspondence.Inthis paperweaddress3D to2Das wellas3D to3Dcorrespondence.

Toreconstructa3Dshape ofaf acefroma 2Dimage,

BlanzandV etter[7]introduced the3DMM.WiththeBasel

FaceModel[23],a3DMM hasbeenmade available tothe

basedonlocal featuresthatpro videsaccuratereconstruc- tions.Acommon andsignificantdra wbackofthe 3DMM isitslack ofrobustness inthecase of"faces inthewild", especiallyifthe faciallandmarks arenotperfectly detected. AlthoughBreueret al.[11]propose tousea SupportVec- torMachinefor automatic3Df acereconstructionand in [10]anidea ispresentedto correctmisplaced landmarksto someextent, bothimplementationswerenotrob ustenough tohandledif ficultscenarioscaused byfacialexpressions orcomplex lightingconditions.Withtheapproach inthis paper,weaimtoo vercomethe previousdra wbacksofthe 3DMM.

Additionallythereare approacheswhichare notaiming

atthereconstruction offaces directly,b utprovide astrong foundationforfurther processingbydetecting faces,esti- matingposes,localizing featurepointsor aligningface ge- ometries[33,32, 25,31,17].

3.3DMor phableModel

The3DMorphable Model[7]is avector spaceof3D

shapesandte xtures,Si=(X1,Y1,Z1,X2,...,Zn)Tand T andr,g,bcolorsofn=113753 vertices.Inourexper - iments,the3DMM isconstructedfrom 3Dscansof 200 individualsandfrom35additional scansthatsho wfacial expressionsofasingleindi vidual[6].On theindividual shapes,thee xpressionsandthe textures,aPCAdefines eigenvectorssi,uiandti,respectiv ely,andaverageshapes andtextures sandt.Inthis basis,new facescan beapprox- imatedbylinear combinations S= s+m? i=1α isi+p? i=1γ iuiT=t+m? i=1β iti.(1)

Weusem=100eigenvectorsforindividualvariationsa nd

p=4forthemost importantdegrees offreedomof facial 3419
expressions,withafocuson mouthmov ements. thoseinthe database"faces inthewild", involv enon-neutral facialexpressions,soour approachofcombiningmultiple dom.We useseparatePCAsandbasisv ectorsforshape and expressioninordertobe abletogi vethe 3Dfaces neutral expressions(γi=0)afterfitting.

3Dshapereconstruction byfittingthe modeltoan image

isessentiallya minimizationofthe imagedistance d image=? u,v?Iinput(u,v)-Imodel(u,v)?2(2) inall3 colorchannels,with respecttothe linearcoefficients lightingandother parameters(fordetails see[7]). Overfittingisavoided byare gularizationtermthatisthe

Mahalanobisdistancefrom thestartingconditions,

d maha=? iα 2i

σ2S,i+?

iγ

2iσ2S,i+?

iβ

2iσ2T,i+?

i(ρi-

ρi)2

σ2R,i,

(3) where ρidenotesthestarting valuesof therenderingparam- eters,andσarethestandard deviationfrom PCA.

Astochasticne wtonoptimizationalgorithm minimizes

theweightedsum ofdimage,dmahaandanadditional term d features=? j??xj y j? -?Px(Xkj,Ykj,Zkj) P y(Xkj,Ykj,Zkj)? 2 (4) whichisthe sumofsquared distancesbetween2D feature positionsxj,yjandtheprojected positionsofthe corre- spondingverte xkj,witha perspective projectionP[8]. d featuresisonlyfor initialization,witha weightthatde- creasesasthe fittingproceeds.

Unlikeearlierworkon 3DMMfitting[8], weuseafea-

turedetectionalgorithm byZhuand Ramanan[33]for an automatedprocess. Thereducedprecisionofthesefeatures andthesuboptimal choiceoffeatures (silhouettes,ears)af- fectthequality oftheoutput significantly.In theremainder ofthispaper wedescribeho wtoselect themostsuccessful reconstructionsbasedon agiv ensetof imagesofa person, andhow tocombinethesetoa3D face.

4.QualityMeasur es

Forameaningfulqualitymeasure, itisimportant tobe

independentoff acialexpression. Therefore,weuse“neu- tralized"facial expressions(withγi=0)inthis sectionex- ceptforimage distance.Becauset heimage distancecom- parestherendered reconstructionwith theoriginalinput im- age,itneeds tobeas closeaspossible totheoriginal face.

Figure2:The imagedistanceis computedbysubtracting

theinputimage withamodified versionwhere therecon- structedface isrenderedontopofthe originalface.

4.1.ImageDistance

Incontrastto allothersdistance functionsthatare dis- cussedinthis paper,the imagedistancedimageEq.(2)is theonlyone thatpenalizesdif ferencesbetweenthe original faceandthereconstruction.The otherdistancemeasures willonlyestimate theplausibilityof naturalnessof recon- structedfaces. Fig.2illustrates onemajordra wbackofthis errorfunc- tion:itis notpossibleto penalizethef actthatthe projected facedoesnotoccludethe completeface intheinput image. Thisisthe caseforObama" srightear .InIinput-Imodel, theimagedistance formostpix elsofthe rightearis zero andthereforethe errorisquite small.Thereconstructed ear isrenderedon thecheek,b utdueto thesimilarcolor ,this hasalsolittle effecton dimage.Ingeneral, dimagefailsto capturesmallb utrelev anterrorsandartifactsin therecon- struction.

Ontheother hand,dimagecanalsobe highev enthough

thefaces looksimilar,forexample whentheo verallcolor toneiswrong orthef aceisslightly shifted.Allof these problemsarecaused bythef actthatdimageisasum ofall pixelsandthatmany smallerrorscount morethana few largeerrors.

Eventhoughdimageturnsoutto besuboptimalfor rating

thequalityor plausibilityofthe 3Dreconstruction,as we willdemonstratein Section5,it makessense tousedimage inthefitting procedurebecause,unlik ethefollo wingcrite- ria,itmeasures thedistancefrom theinputf ace,andit is easytocompute.

4.2.MahalanobisDistance

Mahalanobisdistancemeasures thedistanceof thecur-

rentsolutionfrom theav erageface usingPCA,taking into accountthestandard deviationsobserv edinthe training data.Itis directlyrelatedto themultiv ariateGaussianprob- abilityde nsityfunctionwhichisestimatedbyPCA.Justas theimagedistance, theMahalanobisdistance isalreadyin- tegratedinthe3DMMfitting procedure.For theexperi- mentsinSection 5,wherewe onlywant toratethe quality ofthereconstructed shape,wesimplified Eq.(3)to d maha=? iα 2i

σ2S,i,(5)

3420
sowemeasure onlythedistance oftheneutral faceshape fromthea veragef ace,whileexpressions,textureandren- deringparametersare omitted.Themoti vationis that,un- likeneutralshape,thete xtureande xpressionofa successful reconstructionmaybe farfrom theav erageiftheinputim- ageisunusual (hair,f acialhair, eyeglasses,smile).

4.3.EuclideanDistance

fromthea veragef aceistheEuclideandistancebetweenthe reconstructedshapev ector(withneutralized expression)S, andthea veragev ector s: d eucl(S, s)=???? 3n? i=1(Si- si)2=||S-s||2.(6) Pleasenotethat deuclissensitiv etorigidtransformations ofthef aces.The3DM Mshapevectorsare,by construc- tion,aligned inaleast-squaressense.In3DMM fitting, rigidtransformationsare appliedtothese externally, and capturedbyrendering parametersρi(Section3).Still, a generaldrawback ofdeuclremainswithrespect tosimple, globaltransformations,e.g. anisotropicscaling, whichdoes notaffect naturalnessorshapesimilarity,b uthassignificant effectondeucl. Equation(6)tends toov errateoutlierv erticesinthe sum ofsquareddistances. Forthe evaluation (Section5),we alsoconsidereda modifieddistancewhich isthesum of3D vertexdistances(squarerootonaper -vertex level). Butwe foundnoimpro vement,so Section5willrefertoEquation (6)only.

4.4.NormalDistance

Wehaveobserv edthatlocalorevenglobaldistortions

ofthesur faceare acommonfeatureoffailed3Dface re- constructions.Thisis trueformost orperhapsall 3DMM algorithms(seeSection 2)and- inadif ferentcontext - evenfor3Dshapecapturesetupssuch asscannersor stereo andmultivie wtechniques.Forshapefittingalgorithms,it isunlikely thatafailedreconstructionis misalignedand stillcloseto theav erage,becausemisalignments tendto haveundesiredeffectsonthecost functionsofthe fitting algorithmandtherefore leadaw ayfrom thesetof plausi- blefaces. Inourcontext,misalignmentsmay becaused byinaccurateinitial featurepositions.Also, otherpotential reasonsforf ailedreconstructions,such aslightingeffects, occlusionsore xtremefacial expressions,tendtoleadthe algorithmfar awayfromthea verage,andaverysensitive measureforthis isthede viationofsurf acenormalsfrom theav erage.

Wewouldlike topointoutthatregularization mecha-

nisms,suchas Equation(3)reduce thiseffect andkeep the (a) (b) (c) Figure3:The Normaldistanceis determinedbycomputing theanglebetween thenormalof theav erage(Fig.3a) and thereconstructed face(Fig.3b)percorresponding vertex pair(seeFig. 3c).Thesev aluesarea veragedper segment (seeFig.4a) orface toobtaina globaldistancev alue. solutioncloseto theav erage.Still,for practicalpurposes, wehav eobservedthat(1)iftheweightof theregulariza- tionistoo large,it impliessuboptimalresults onimagesthat wouldotherwisebereconstructedsuccessfully ,sothere isa fundamentaltradeoff betweenqualityandrobustness,and (2)there gularizerEq.(3) isnotareliablemeasureof plau- sibilityoff aces,aswe willseeinSection5.

Basedonthe densepoint-to-pointcorrespondence be-

tweenvertices iofthe3DMM, thenew distancemeasure d normalanalyzesthedif ferencebetweenthe surfacenor- malsniofthereconstr uctedface, andthenormalsn?iof theav erageface: d normal=1 nn i=1arccosni·n?i?ni??n?i?.(7) Theideaof thisNormaldistance isillustratedin Fig.3. Notethat,unlik edeucl,dnormalisinsensitiv etoscalingand shifting.Byse gmentingthefull faceintodistinctfacial re- gions(eyes, mouth,noseandsurroundingregion, seeFig.

4a),separatedistances dnormalcanbedefined thatreflect

theplausibilitiesof regionsseparately .We willusethisidea inSection6.

Inhumanfaces,thenormalsinsomeverticesonthe nose,

theeyesorthelipsvarymorethanothers.W ehaveanalyzed theoriginal200 3Dscansof the3DMMand createddiffer - entweightmaps ω(seeFig.4b) whichaccountfor these localdifferences byscalingregionswithhigh normalvari- ationeitherup (consideringthemmost diagnostic)ordo wn (normalization).Ina firststep,wecomputedtheaveragede- viationangle

φiofthenormal nifromthea veragenormal

n ?iineachv ertexiacrossall200 faces.Then, wefoundthe bestweightmap tobedefined byˆωi=1-

φi-φmin

φmax-φmin,and

theweightedNormal distanceis d normalW=1 nn i=1ˆωiarccosni·n?i?ni??n?i?,(8) forwhichwe obtainedexperimental resultsthatare summa- rizedinthe nextsection. 3421
(a) (b) Figure4:Fig. 4ashows thedifferent facese gments.The

3DMMbasedweight mapissho wninFig. 4b.

5.Evaluating theDistanceMeasures

Thegoalof thisev aluationisto findoutwhich quality assigntodif ferentreconstructions.F orhumans,quality maymeanho wnaturaland plausiblethe3Dfacelooks, butalsohowsimilar itisto thepersonintheimage.F or failedreconstructions,bothcriteriaare usuallyviolatedat thesametime, sothedistance measuresfromSection 4are goodcandidatese venthough mostdonotmeasuresimilar- itytothe inputface.

5.1.Evaluation 1

Thefirstranking wasperformed on243D reconstruc-

tionsfrompictures ofBarackObama basedonautomati- callydetectedlandmarks. Theautomaticdetection ofland- marksisbas edonthe approachofZhuandRamanan[33]. Anadditionalset of24reconst ructionswas createdbyus- ingmanuallyselected landmarkson thesameinput images. Againthealgorithmicdistance measuresintroducedin Sec- tion4were usedtoperform aranking.All reconstructions werecreatedfrom asingleimage asdescribedin Section3.

Weaskedfournai veparticipantstocreatea rankingin

eachofthe twosets of24reconstructions, basedontheper- ceivedqualityofthereconstruction.Theindi vidualuser rankingswerecombined todefinean overall rankinglist, whichwas comparedtotherankingofeach distancemea- sure.Ascan beseenin Table1, themeanand maxerrors (differenceofranksassignedto eachreconstruction)of Ma- halanobisandNormal distancearemuch lessthanthe ones onthenumbers fordnormalW(seeEq.8), itcanbe noted thattheinfluence oftheweight mapisnot verystrong com- paredtothe rankingbasedon dnormal(seeEq.7).

InFig.5 thecorrelationof eachdistancemeasure is

visualized:Thehorizontal axisdescribesthe average user ranking,whileeach distancemeasureis mappedtothe ver- ticalaxis.If adistancemeasure correlatesperfectlywith theuserranking, thedotsof thescatter diagramarealigned alongthediagonal. Ascanbe seeninFig. 5a,forthe image distancethedots arewidelyscattered. Thesamecan beob- servedfortheEuclideandistance inFig.5b. Consequently, (a) (b) (c) (d) Figure5:V isualizationofthe correlationbetweentheav- erageuserranking andeachdistant measure(100?=very good,0?=verybad)forreconstructionsbased onautomatic (red)andmanual(yellow)landmarkselection. bothmeasuresare notusefulto distinguishplausiblefrom implausiblereconstructionsin away thatcorrelatesto the opinionofusers. Forthe Mahalanobis(seeFig. 5c)andthe Normaldistance(see Fig.5d),the correlationbetween the userratingand theratingbased onthealgorithmic distance measuresisclearly visible.Especiallythe Normaldistance predictsmuchof thequalityjudgments ofourparticipants. Thecorrelationsfor all2·24reconstructions(automatic andmanual)are fordimage:0.27,for deuclidean:0.27,for d maha:0.85and fordnormal:0.94.

5.2.Evaluation 2

Inasecond evaluation, 3Dreconstructionsbas edonim- agesofObama (24images),La wrence(32images), Annan

Obamadataset

automaticlandmarksmanuallandmarks meanerrormaxerrormeanerrormaxerror dimage6.92196.8318 deucl5.67168.0820 dmaha2.2583.4212 dnormal1.3352.256 dnormalW1.3352.176

Table1:Meanandmax differenceof ranksfor24 recon-

structionswithautomatically and24with manuallyselected landmarksbasedon theperceiv edqualityof fournaiv epar- ticipants(seeSection 5.1). 3422

(32images),W atson(46images) andCarell(28images)wererated.Ag aintwo distinctsetswerecreated,but thistimeonlyautomatically selectedlandmarkswere utilized.Thefirstset wascreated byfittingto asingleimage,whileforthesecond setasimultaneous fittotw oimagesw asper- formedbyapplying themultifitapproach ofBlanzand Vet-ter[7].A fixedreference imagewas selectedandwasthencombinedwitheach otherimageof thecollectionfor theperson.Pleasenote thatthef aciallandmarksdif ferfromto theonesin Section5.1.Thus, althoughthesame inputim-agesareused fortheObama dataset,thereconstructions aredifferent.

Foreachdataset,thedistance measureswereused to

createaranking list.Thenwe askedse vennai vepartici- pantstorate each3Dreconstruction. Possibleratingswere "verygood","good","acceptable"or"failed". Theindivid- ualratingswere averaged andthenused tocreatearank- ing.Many reconstructionsobtainedthesameav eragerat- ingsandtherefore manypositions intheranking areshared. Thisimplieshigher discrepanciesbetweenthe ranklistde- uniquerankingdirectly .Still,the Normaldistancematches theuserrating best,ascan beseenin Table2 and3.

ObamaLawrenceAnnanWatsonCarell

dimage5.29(13) 7.56(20) 9.59(30) 13.15(29) 8.04(18) deucl8.38(20) 8.38(22) 10.84(23) 13.54(35) 8.82(20) dmaha5.79(13) 7.00(18) 5.22(16) 11.94(27) 3.46(9) dnormal5.21(13) 5.44(14) 4.97(12) 11.50(27) 2.61(8) dnormalW5.21(13) 5.31(14) 4.97(12) 11.41(27) 2.46(8) Table2:Meanandmax (inbrac kets)differenceofranksfor reconstructionsfroma singleimagebased onthe perceived qualityofse vennai veparticipants(seeSection5.2).

ObamaLawrenceAnnanWatsonCarell

dimage6.75(14) 8.34(23) 8.75(24) 10.80(33) 6.32(19) deucl5.58(16) 8.47(24) 6.56(19) 10.02(34) 9.32(24) dmaha4.33(11) 5.47(14) 5.25(19) 10.07(28) 6.82(19) dnormal4.08(10) 4.41(14) 4.31(15) 7.85(25) 4.96(17) dnormalW4.08(10) 4.41(14) 4.31(15) 7.80(25) 4.96(17)

Table3:Meanandmax (inbrac kets)differenceofranks

forreconstructionsfrom multipleimagesbase donthe per- ceivedqualityofsevennai veparticipants (seeSection5.2).

6.Weighted LinearCombinationperSegment

Theautomated3D reconstructionthatwe proposeinthis

papercompensatesthe reducedprecisionand reliabilityof automaticallydetectedfeature positionsbyusing morethan asingleimage ofthef ace.Notethat, unlikestereo andmul- tiviewalgorithms,weallowfornonrigid deformationsdue tofacial expressions,andlargedif ferencesinthe(unknown) imagingconditions. Figure6:Plausibility ratingofthe singleimagebased re- constructionsusingNormal distancewithsubsequent order- ing.

Ourstrategy istoapplysingleimage3DMM fitting(Sec-

tion3)on eachofthe inputimagesof thepersonseparately , basedonlandmarks detectedbythe algorithmbyZhu and

Ramanan[33],select thembestresults(Fig. 1and6) on

eachsegment (Fig.4a)usingdnormal,computeweighted linearcombinationsof theseandmer getheminto asingle

3Dface.

Theshapefor eachsegment isdeterminedby aweighted

linearcombination ofcorrespondingsegmentsbasedon the rankinglistorder .Theweight decreaseswiththetherank.

Thusthecombined shapeforeach individualse gment

S seg=m-1? i=0α iSseg,i(9) isdeterminedby mindividualreconstructionsofcorre- spondingsegments Sseg,iweightedbyquotesdbs_dbs17.pdfusesText_23