[PDF] La fin des frontières entre réel et virtuel: vers le monisme numérique

View - Download La fin des frontières entre réel et virtuel: vers le monisme numérique

Previous PDF

Next PDF

IEEE TRANSACTIONSONVISU ALIZATION ANDCOMPUTERGRAPHICS,VOL. 14,NO .8, AUGUST20201

Visual Qualityof 3DMeshes withDiffuse Colors

in VirtualReality:Subjectiv eand Objective

Evaluation

YanaNehm

´e, FlorentDupont, Jean-PhilippeF arrugia, PatrickLeCallet, Fellow,IEEE and GuillaumeLa vou

´e,Senior Member,IEEE

Abstract

—Surfacemeshes associatedwith diffusete xtureor colorattr ibutesarebecoming popularm ultimediacont ents.Theyprovide

a highdeg reeofrealismand allow sixdeg reesof freedom(6DoF)interactions inimmersiv evirtualreality environments .J ustlik eother

types ofm ultimedia,3Dmeshesare subjectto awide range ofprocessing, e.g., simplication andcompression, whichresultina loss

of qualityof thenal renderedscene .Thus ,both subjectivestudiesand objectivemetr icsare neededtounderstandand predictthis

visual loss.Inthis wor k,w eintroducealargedatasetof 480animatedmesheswith diffusecolor infor mation,and associatedwith

perceivedquality judgments. Thestimuliw eregener atedfrom5source modelssubjectedtogeometr yandcolordistor tions. Each

stimulusw asassociatedwith6 hypothetical rendering trajector ies(HRTs):combinations of3viewpoints and2animations.Atotal of

11520 qualityjudgments (24per stimulus) were acquiredinasubjectiv eexperiment conducted invir tualreality .Theresultsallo wedus

to exploretheinuence ofsource models, animationsand viewpoints onboththequalit yscores andtheir condenceintervals .Based

on thesendings ,wepropose therstmetric for qualityassessment of3D mesheswithdiffusecolors ,whichwor ksentirely onthe

mesh domain.This metric incorporatesperceptually-rele vantcurvature-basedand color-basedfeatures.We evaluateitsperfor mance,

as wellasa number ofImage QualityMetrics(IQMs), ontw odatasets: oursand adatasetofdistor tedte xturedmeshes .Ourmetric

demonstratesgood resultsand abetter stabilitythan IQMs .Finally ,w ein vestigatedhowthekno wledgeofthevie wpoint(i.e.,the visible

partsof the3D model)ma yimpro ve theresultsofobjectiv emetrics.

IndexT erms

—Computer Graphics,Perception,Vir tualreality,Diffuse Color,3DMesh, VisualQualityAssessment,Subjectiv eQuality

Evaluation,Objectiv eQualityEvaluation, Dataset,P erceptualMetric.F

1 INTRODUCTION

A

Stechnological advancesand capabilitiesin theeld

of computergraphics grow daybyday, theneed to master themanipulation, visualizationand processing of

3D digitaldata increases atanequalpace. Indeed,the

development ofmodeling software andacquisitiondevices (3D scan,r econstructionprocess)makes3D graphics(mesh, voxel, pointcloud) richand realistic: complexmodels with millions ofgeometric primitives,enriched withvarious ap- pearance attributes(color ,texture,materiall, etc.).Theway in whichthis 3Dcontent isconsumed isalso evolvingfr om standardscr eenstoVirtual andMixed Reality(VR/MR). However,the sizeand complexityof theserich 3Dmodels often maketheir interactivevisualization problematic. This is particularlythe casein immersiveenvir onments(using head-mounted displays)and/or incase ofonline applica- tions (wherefasttransmission isneeded). Thus,to adaptthe complexity ofthe 3Dcontent forlightweight devicesand to avoidlatency dueto transmission,diverse processing operations, includingsimplication andcomp ression, are usually applied.These processes arelossy. Theyoperateon

both geometryand appearanceattributes, whichinevitably Y.Nehm ´e,F.Dupont,JP .FarrugiaandG. Lavou´ earewith theCNRS,

Univ Lyon,LIRIS,France.

E-mail: yana.nehme@insa-lyon.fr,jean-philippe.farrugia@univ-lyon1.fr, P.Le Calletis withthe CNRS,Univ Nantes,LS2N, France.E-mail: patrick.lecallet@univ-nantes.fr. Manuscript submittedJune 16,2020. introducedistortions thatimpact the perceived qualityof the dataand thusthe qualityof userexperience (QoE). Objective qualitymet ricsarethus criticallyneededto automatically predictthelevel ofannoyance causedby these operations.Most metricsin theliteratur eeval uateonly geometric distortions(i.e. theyconsider mesheswithout appearance attributes),e.g. [1],[2], [3].When itcomes to meshes withdif fusecolorinformation(either inthe form of textureorvertex-colors), littlework hasbeen published [4] [5].Actually ,forthiskind ofdata, itis stillunclear how color andgeometry distortionsaf fectquality .Thereis alack of bothobjective metricsand subjectivedataset s.Another factor thathas notyet beenexplor ed,and whichis relevant in thecase of6 Degrees ofFr eedom(DoF)interactions, is how theviewpoin tandmovementof 3Dmodels affect their perceivedquality . In thiswork, weaddr essthe problemofsub jectiveand objective qualityassessment of3D modelswith diffuse colors. Ourrst goalis topr oducea ground truthdatabase of 3Dgraphics withquality judgments,and tounderstand the impactof severalfactors (suchas thesour cemodels, distortions, viewpoints,and animations)on theper ceived quality ofthese data.The experimentis basedon adouble stimulus impairmentscale (DSIS)method andinvolves 480 animated stimulicr eatedfromve sourcemodels,each cor- ruptedwith colorand geometrydistortions anddisplayed in 3dif ferentviewpointsthatweanimated with2 short

movements. Wechoseto conductthe experimentin Virtual This is the author's version of an article that has been published in this journal. Changes were made to this version by the publisher prior to publication.

IEEE TRANSACTIONSONVISU ALIZATION ANDCOMPUTERGRAPHICS,VOL. 14,NO .8, AUGUST20202

Reality (VR)using theHTC Vive Pro headset,sinceVR

user studiesof ferthemostecological andr ealisticuse cases and arein highdemand.Thisdatabase isused toanalyze the factorsthat inuencethe subjectivequali tyassessment of 3Dgraphics: weevaluate notonly thevisual impactof color andgeometry distortionson theappearance ofsuch data, butalso theimpact ofsour cemodels, animationsand viewpoints. Considering thendings ofthis subjectiveevaluation, we designan objectivequality assessmentmetric forcolor ed meshes:CMDM(Color MeshDistortion Measure). Thisisa full-referencedata-drivenmetric thatfully operateson the mesh domain,at vertexlevel. Itconsists ofa linearcombi- nation ofper ceptually-relevantfeaturesrelated tocolorand geometry.The optimalset offeatur eswas selectedthr ough logistic regression.Weuse oursubjectiveground truth to evaluate theperformance ofCMDMas wellas otherstate- of-the-art imagequality metrics.Mor eover, toassessthe robustnessof our metric,wetestit ona newdataset of texturedmeshes corrupted withcompounddistortionsthat differsignicantly from thoseusedtotrain it.Our metric demonstrates goodr esults.Finally,we studytherelevance of incorporatingthe viewpoint(the visibleparts) ofthe 3D model intoobjective metrics.

Wesummarize ourcontributions asfollows:

1) Weprovide thecommunitywitha ground truth dataset

of 480animated mesheswith vertexcolors, eachrated by 24subjects. Thisdataset isthe largest onefor quality assessment of3D contents withdiffuse colori nformation, and therst basedon vertexcolor repr esentation.It is also therst publicdataset

1producedin VRfor such

data.

2) Weprovide anin-depthanalysis ofthe effects ofsource

models, distortions,viewpoints andmovement onboth mean opinionscor es(MOSs)andcondence intervals (CIs). Ourndings provi deinsightsforthedesignof both subjective studiesand objectivemetrics.

3) Weevaluateindividually theperformance ofa setof

perceptually-relevantcurvature-based andcolor-based featuresfor predicting theperceived visualqualityof coloredmeshes.

4) Wedevelopand learna perceptually-validat edmetric

for measuringthe qualityof colored meshes.T othe best ofour knowledge,our proposed metricis therst attempt tointegrate bothgeometry andcolor informa- tion forquality assessmentof suchdata. Ourmetric demonstrates goodr esultsandstabilityon twodif ferent datasets. Thesour cecodeofthe metricis madepu blicly available

2on theMEsh Processing Platform(MEPP)to

support furtherr esearchinthisarea.

5) Weinvestigatehow knowledgeof theviewpoint may

improver esultsfromobjective metrics. The paperis organized asfollows:section2 provides a reviewof theexisting workson subjectivean dobjective quality assessmentof 3Ddata. Insection 3,we describe the subjectivestudy ,beforepr esentingtheresults insection

4. Section5 detailsthe proposed metric,while section6

presentsits validationas wellas acomparison withstate-

1. https://yananehme.github.io/

2. https://github.com/MEPP-team/MEPP2of-the-art imageand meshquality metrics.The studyon

integrating theviewpoint inobjective metricsis presented in section 7along withits results. Finally, concludingremarks and perspectivesar eoutlinedinsection 8.

2 RELATEDWORK

In thissection, wepr ovidean overviewofexistingdatasets and metricsfor predicting theperceivedvisual impactof distortions appliedtographical 3Dcontent (3Dmeshes and point clouds).W earespecically interestedin3D content with diffusecolors,either inthe formof texture maps or vertex/pointcolors. Notealso thatthis state-of-the-art focuses onthe visualimpact ofdistortions appliedon the3D content itself(e.g. introduced bycompression,simplication or ltering);it doesnot coverthe visibilitypr edictionof artifacts introducedduringthe rendering process (e.g.by global illuminationappr oximation)orafterr endering (e.g. by tonemapping). Adataset hasbeen recently introduced that focuseson thesetypes ofartifacts [6].For amor e complete surveyof theeld ofper ceptionand quality assessment incomputer graphics,we refer ther eaderto[7].

2.1 Subjectivequality experiments anddatasets

As statedin theintr oduction,datasets ofhumanperceptual similarity judgmentsar eofprimaryimportance forunder - standing humanbehavior inevaluating perceive dquality , as wellas fortraining andbenchmarking objectivemetrics. Many authorshav econductedsubjectivequality assessment tests involving3D meshes[2], [5],[8], [9],[10], [11],[12], [13], [14], [15],[16] or3D pointclouds [17],[18], [19],[20], [21]. They consideredavariety ofmethods: AbsoluteCategory Rating (ACR)[11], [13],[22], Double-StimulusImpairment Scale (DSIS)[2], [8],[9], [18],[19], [21]and PairwiseCom- parison (PC)[5], [14],[15], [23].V eryr ecently, astudy[16] attempted tocompar ethesesubjectivemethods andshowed that, forthe particularcase of3D graphicalcontent, the DSIS methodtends topr oducemor eaccurateresults than ACR (i.e.,MOS withsmaller condenceintervals). Existing subjective experimentsalso considered differentways of presenting3D content:static images[8], animatedcontent without interaction(usually low-speedr otation)[5], [9],[10], [16], [19],[20], [21],[22], [23]or interactiveconte nt[2], [11], [13], [14],[15], [17],[18]. We denotethat onlyin[15],[16], the experiments wereconductedin aVR environment. Sofar ,no attempts havebeen madeto fullyunderstand theimpact of these designchoices onthe obtainedmean opinion scores and theiraccuracy . Unfortunately,among theworks presented above, very few havepublicly released theirdatasets.For3D meshes,the available datasetsof meanopinion scores concernmostly geometry-only content[11], [12],[14] andar eall rather small (resp.88,26 and30 models).The onlypublic datasets involving 3Dmeshes withdif fusecolor informationare providedby Guoet al.[5] andfr omZerman etal. [22],and contain respectively136and 28stimuli. Forboth cases,the color informationis provided astexture maps.For colored point clouds,the available datasetsarethose provided by Javaheri etal. [20],Alexiou etal. [18]and Zermanet al.[22],

and containr espectively54,244and 136stimuli. Notethat This is the author's version of an article that has been published in th

is journal. Changes were made to this version by the publisher prior to publication. IEEE TRANSACTIONSONVISU ALIZATION ANDCOMPUTERGRAPHICS,VOL. 14,NO .8, AUGUST20203 the datasetfr omZermanetal. [22]actually containsboth meshes andpoint clouds,for atotal of164 stimulithat were rated inthe same subjectivetest.Allthese datasetswer e generated throughexperimentsconducted onscr een.

In thiswork, wepr oposea datasetof480animated

meshes withvertex colors.It isthe largest onefor quality assessment of3D contentwith diffuse colorinformation, and therst basedon vertexcolor repr esentation.Note that themeshrepresentationdiffers considerablyfromthe point cloudrepresentationinseveral aspectssuch asthe way they arerender ed,andthenatureof commonlyapplied processingoperations (andthus distortions).Our dataset allows usto prov ideaninitialinvestigationonthe inuence of movementand viewpointon thequality evaluationof

3D content.As in[15], [16]we considered aVR contextand

we useda DSISmethod with24 observersper stimulus,as recommendedin [16].

2.2 Objectivequality metrics

Inspiredby thevast amountof previous workson im-

age andvideo qualityassessment, severalobjective quality metrics havebeen introduced for3Dmeshes.These are mostly full-reference(comparethe distortedmodelwithits original/reference)andfollow theclassical approach used in imagequality assessment:local feature differ encesare calculated atvertex level,which are thenpooled overthe entire3D modelto obtaina globalquality score. Existing metrics relyonvarious geometrycharacteristics: curvature [1], [24],dihedral angles[2] orr oughness[3], [13].A survey [25] showedthat MSDM2[1], FMPD[3] andDAME [2]ar e excellent predictorsofvisual quality. Very recently,several authors proposeddata-drivenappr oachesbased onmachine learning [26],[27]. Besidesthese workson globalvisual quality assessment(suited forsupra-thr esholddistortions), Nader etal. [28]intr oduceda bottom-upvisibilitythreshold predictorfor 3Dmeshes. Guoet al.[29] alsostudied thelocal visibility ofgeometric artifactsand showedthat curvature could bea goodpr edictorof distortionvisibility.

The above-presentedworksonly takegeometry intoac-

count. Withrespect to3Dcontent withcolor ormaterial information, veryfew workshave beenpublished. For meshes withdif fusetexture,T ianetal.[4] andGuoetal. [5] proposedmetrics basedonaweighted combinationof a global distanceover geometry(Mean Squared Error (MSE) over meshvertices in[4], andMSDM2 in[5]) anda global distance overtextur eimage(MSEover texture pixels in[4], and SSIMin [5]).While thelatter metricdemonstrated good resultson asubjective datasetof distortedtextur edmeshes [5], combiningerrors computedondiffer entdomains (3D mesh andtextur eimage)maybe hazardous sincemany external factors(e.g. texelsize, visibilityof differ entparts) may impactthe results.

Withr egardtothisprevious work[4], [5],we propose

a data-drivenmetric thatfully operateson themesh do- main, atvertex level.W econsider aninitialcollectionof perceptually-relevantfeatures relatedtocolor andgeometry. These featuresare takenfromexisting workso nquality assessment of3D meshes[1] andcolor images[30]. Asubset of thesefeatur esisthenoptimall yselected andcombined,

based onthe re sultsofoursubjectivestudy. Ourmetric providesexcellent results anddemonstratesagood stability, both formeshes withvertex colorsand fortex tured meshes.

For qualityassessment of3D colored pointclouds, a

data-driven metric(PCQM) hasbeen recently introduced by Meynet etal. [31].Our metricconsiders thesame initial col- lection ofcolor andgeometric features as[31]. Nevertheless, moving frompointcloud domainto meshdomain implies major adaptationsin thecomputation ofthese features. Other differencesbetweenourmetric andPCQM are: the optimal selectionand combinationof features, themulti- scale approach,andthe viewpointintegration mechanism. Our metricis alsor elatedto theworkofV anhoeyet al. [23], whopr oposedaqualitymetric forsurface light-elds (i.e., per-vertexdirectional color).However,thei rmetric considers colorinformation onlyand isactually asimple

MSE overboth thedir ectionaland spatialdomains.

All themetrics presented abovearemodel-based, i.e., they operateon the3D modelitself (orits attributes like texturemaps). However ,toevaluatethequalityof 3Dcon- tent, severalauthors havealso considered ImageQuality Metrics (IQM)computed onr endered snapshots.Forex- ample, Yangetal. [32]and Caillaudet al.[33] respectively used imageMSE andSSIM [34]to optimizetextur edmesh transmission. Theadvantage ofimage-based metricsover model-based metricsis theirnatural abilityto handlethe multimodal natureofdata (geometryand coloro rtextur e information), aswell astheir naturalincorporation ofthe complex renderingpipeline(computation oflight material interactions, viewpointselectionand rasterization).On the other hand,IQMs poseother problems: (1)it isnecessaryto know inadvance thenal rendering ofthe stimuliinorder to predicttheirquality withthese metrics(because IQMs operate on2D render edsnapshots).(2)IQMsalsoneed the knowledge ofthe displayedviewpoint. Usingthe min a view-independent wayintr oducesnewparameterssuch as choice ofthe 2Dviews, orpooling ofquality scores obtained fromdif ferentviewsintoasingle globalscor e.(3) IQMsar e not practicalfor drivingpr ocessingoperations (e.g.mesh simplication). Model-basedmetricsar ebetter suitedfor these operationssince theyoperate onthe meshdomain, i.e. thesame repr esentationspaceasmeshprocessing algo- rithms. Thismakes itpossible todrive apr ocessglobally (on the entiremesh)as wellas locally(at vertexlevel). (4)Recent studies aboutthese view-basedappr oaches[5], [35]tendto show thattheir performancegr eatlydepends ondistortions and contents,and fallwell behindmodel-based approaches.

3 SUBJECTIVEEXPERIME NT

Weconducted alar ge-scalesubjective experimenttoevalu- ate thevisual impactof colorand geometrydistortions on the appearanceof colored 3Dmodels.Our datasetcontains

480 animated3D modelscr eatedfr omverefer enceobjects,

on whichar eappliedfourtypes ofdistortion andtwo types of animation.This datasetextends theone presented in[16] composed of80 stimuli.The subjectivestudy wasconducted in avirt ualrealitysetting usingaDSISmethod. Thissection

providesdetails onthe subjectivestudy .This is the author's version of an article that has been published in th

is journal. Changes were made to this version by the publisher prior to publication. IEEE TRANSACTIONSONVISU ALIZATION ANDCOMPUTERGRAPHICS,VOL. 14,NO .8, AUGUST20204 Ari_VP2 Ari_VP1 Ari_VP3 Samurai_VP2 Samuai_VP1 Samurai_VP3

Chameleon_VP3

Chameleon_VP2

Fish_VP3

Fish_VP2

Dancing

Drummer Fig. 1:Illustration ofthe 3Dgraphic source modelsand theirselected viewpoints,respectively. Acronyms refer to

Model

3.1 Stimuli

3.1.1 3Dsource modelselection

Tobuild ourdataset ofcolor ed3D models,we selected5 high-resolutiontriangle meshes,each havingdi ff usecolor information representedbyvertexcolors (note xture map- ping). Thesemodels were chosensoasto ensure avariety of shapesand colors.T able1 detailsthecharacteristics ofthe models, whileFigur e1illustratesthem. Notethat, thesixth model ("

Dancing Drummer

") isnot partof thedataset. Itwas used atthe trainingstage ofthe experiment. TABLE1: Characteristicsof the3D graphicsour cemodels Models#Vertices

Geometry

complexityColor characteristicsSemantic categoryCreated using

Aix686061

Plane with

small detailsMono-color Art3D scanning

Ari 645492Intermediate

Cool &light

colorsHuman statues3D scanning

Chameleon 588441

High &sharp

edgesCool &dull colorsAnimalModeling software

Fish 216578

Low &sharp

edgesCool &warm colorsAnimalModeling software

Samurai 449997High warmcolors

Human statues3D scanning

Dancing

Drummer1335436Intermediate/

HighCool colorsHuman

statues3D scanning3.1.2 Distortions The sourcemodelspr esentedabove havebeencorrupted by the following4 typesof distortionapplied ongeometry and color.These selected distortionsrepresent commonsimpli- cation andcompr essionoperationstypicallyused in3Dquotesdbs_dbs46.pdfusesText_46

[PDF] Le virus du chikungunya

[PDF] le visage de mae west salvador dali wikipedia

[PDF] le vivant est-il un objet de science comme un autre

[PDF] le vivant et le non vivant

[PDF] le vivant et son évolution

[PDF] le vivant et son évolution controle

[PDF] le vivant peut-il être considéré comme un objet technique

[PDF] le vivant philosophie fiche

[PDF] le vivant philosophie terminale s

[PDF] Le vocabulaire de Christmas (xmas)

[PDF] le vocabulaire de l aventure

[PDF] le vocabulaire de laventure

[PDF] Le vocabulaire de la divisibilité

[PDF] Le vocabulaire de la lettre

[PDF] Le vocabulaire de la lettre: expressions