[PDF] RankIQA: Learning From Rankings for No-Reference Image Quality

View - Download RankIQA: Learning From Rankings for No-Reference Image Quality

Previous PDF

Next PDF

RankIQA:Learning fromRankingsforNo-r eferenceImageQualityAssessment

XialeiLiu

ComputerVision Center

Barcelona,Spain

xialei@cvc.uab.esJoostvan deWeijer

ComputerVision Center

Barcelona,Spain

joost@cvc.uab.esAndrewD.Bagdanov

MICC,Univ ersityofFlorence

Florence,Italy

andrew.bagdanov@unifi.it

Abstract

Weproposeano-r eferenceimage qualityassessment

(NR-IQA)approac hthatlearnsfromrankings(RankIQA). Toaddressthepr oblemoflimitedIQAdatasetsize ,wetr ain aSiameseNetwork torank images intermsof imagequal- itybyusing syntheticallygener ateddistortionsfor which relativeimagequality isknown.Theserankedimage sets canbeautomatically generated withoutlaborioushuman labeling.Wethen usefine-tuningtotransferthe knowl- edgerepresentedin thetrainedSiameseNetworktoatradi- tionalCNNthat estimatesabsoluteima gequality fromsin- gleimag es.Wedemonstratehowourapproac hcanbemade significantlymore efficientthantraditionalSiamese Net- worksbyforwar dpropa gatingabatchofima gesthrough asinglenetwork andbackpr opagatinggr adientsderived fromallpairsof images inthebatch.Experimentson the TID2013benchmark showthatweimprove thestate-of-the- artbyo ver5%.Furthermor e,ontheLIVEbenc hmarkwe niquesandthat weeven outperformthestate-of-the-art in full-referenceIQA(FR-IQA)methodswithouthavingto re- sorttohigh-quality reference images toinferIQA.

1.Introduction

Imagesaree very whereinourlife.Unfortunately,they

areoftendistorted bytheprocesses ofacquisition,transmis- sion,storage,and externalconditions likecamera motion.

ImageQualityAssessment (IQA)[

34]isa techniquedev el-

opedtoautomatically predicttheperceptual qualityofim- ages.IQAestimates shouldbehighly correlatedwithqual- ityassessmentsmade byarange ofvery manyhuman eval- uators(commonlyreferred toasthe MeanOpinionScore (MOS)[

27,22]).IQAhas beenwidelyapplied toproblems

13], imagesuper-resolution [

32],andimage retrieval [37].

IQAapproachesare generallydivided intothreecate-

goriesbasedon whethertheundistorted image(calledref- erenceimage)or informationaboutit isav ailable:full- Gblur

JPEGLarge ranking dataset

Siamese Network

RankingShared weights

Small IQA dataset

Scores: 36.9 Scores: 26.7

Scores: 60.0

Fine-tuning

QualityDistorted

images

Normal Network

Classical approach

Our approach

Figure1.The classicalapproachtrains adeepCNN regressordi- rectlyontheground-truth. Ourapproachtrains anetwork froman imagerankingdataset.Theserank edimagescan beeasilygener- atedbyapplying distortionsofv aryingintensities.The network parametersarethen transferredtothe regressionnetw orkforfine- tuning.Thisallo wsforthe trainingofdeeperandwidernetw orks. referenceIQA(FR-IQA), reduced-referenceIQA(RR-

IQA),andno-reference IQA(NR-IQA).Research has

mostlyfocussedon themorerealist scenarioofNR-IQA wheretheimage qualityofan imagewithoutan yreference imagehasto beestimated.In NR-IQA,many methodsfo- cusona specificdistortion[

38,7],whichlimits theappli-

cabilityofthese methods.Othermethods considerarange ofdistortions[

20,23,18,19].

ConvolutionalNeuralNetworks(CNNs)areha vingan

enormousimpacton computervisionresearch andpractice.

Thoughthey havebeenaroundfor decades[

16],itw asn"t

until2012,when Krizhevsky etal.[

14]achiev edspectac-

ularresultswith aCNNin theImageNetcompetition, that theyachievedwide attentionandadoptioninthebroader computervisioncommunity .Thearchitectures ofnetworks aregettingdeeper anddeeperwith respecttothe original

AlexNet,withResNetbeingan exampleof verydeep net-

workarchitecture[

8].Theresult ofthistrend isthatstate-

of-the-artCNNslik eAlexNet andResNethavehundred of 1 1040
millionsofparameters andrequiremassi veamounts ofdatatotrainfrom scratch(withouto verfitting).

Thesuccessof CNNsencouragedresearch exploring

theirpotentialapplication totheNR-IQA problem.This researchresultedin significantimprov ementscomparedto previoushand-craftedapproaches[

17,11,12].Themain

problemsthesepapers hadtoaddress istheabsence oflarge datasetsforIQA .Especiallyas networksgro wdeeperand wider,thenumberofparameters increasesdramatically .As aconsequence,lar gerandlar gerannotateddatasetsarere- quiredfortraining. Howev er,the annotationprocessfor IQAimagedatasets requiresmultiplehuman annotations labor-intensiveandcostly.Asaresults,mosta vailableIQA datasetsaretoo smalltobe effectiv efortraining CNNs.

Weproposeanapproachto addresstheabsence oflarge

datasets.Themain idea(seeFig.

1)isthat whilehuman-

annotatedIQAdata isdifficult toobtain,it iseasyto gen- erateimagesthatare rankedaccordingtotheir imagequal- ity.Thatis,wecan generateimages etsinwhich, though wedonot have anabsolutequality measureforeachgen- eratedimage,for anypairofimageswe knowwhich is ofhigherquality .We callthislearningfrom rankingsap- proachRankIQA,and withitwe learntorank imagein termsofquality usingSiameseNetw orks,andthen we transferknowledge learnedfromrankedimagesto atradi- tionalCNNfine-tuned onIQAdata inorderto improve the accuracyofIQA.Theidea tolearnIQA featuresfromdis- tortedreferenceimages wasproposed byZhanget al.ina patent[

44].Inthis paperwego beyondthis patentin that

weprovide adetaileddescriptionofourmethod andex- perimentallyverify theusefulnessofpre-trainingnetworks usingranked datasets.

Asasecond contributionwe proposeamethod foref-

ficientbackpropagation inSiamesenetworks.Themethod forwardsabatchofimages throughasingle networkand thenbackpropagates gradientsderivedfromall pairsinthe batch.Ine xtensiveexperimentsonestablishedIQAdatasets weshow thatlearningfromrankingssignificantlyimpro ves results,andthat ourefficient backpropagationalgorithm al- lowstotrainthesenetw orksbetterand fasterthan other trainingprotocols,lik ehard-neg ativemining.

2.Relatedw ork

Webrieflyreview theliteraturerelated toourapproach.

Wefocusondistortion-genericNR-IQA sinceitis more

generallyapplicablethan theotherIQA researchlines. IQAcanbe classifiedintoNatural SceneStatistics(NSS) methodsandlearning-based methods.InNSS methods, theassumptionis thatimagesof differentquality varyin thestatisticsof responsestospecific filters.Wa velets[

20],DCT[

23]andCurv elets[18]arecommonly usedtoe xtract

thefeaturesin differentsub-bands. Thesefeaturedistrib u- tionsareparametrized, forexample withtheGeneralized

GaussianDistribution [

26].Theaim ofthesemethods isto

estimatethedistrib utionalparameters,from whichaquality assessmentcanbe inferred.Theauthors of[

19]proposeto

extractNSSfeaturesinthe spatialdomainto obtainsignif- icantspeed-ups.In learning-basedmethods,local features areextracted andmappedtotheMOSusing, forexample,

SupportMachineRe gressionorNeural Networks[

4].The

codebookmethod[

40,39]combinesdif ferentfeaturesin-

steadofusing localfeaturesdirectly .Datasetswithout MOS canbee xploitedtoconstruct thecodebookbymeansofun- supervisedlearning,which isparticularlyimportant dueto ofthesmall sizeofe xistingdatasets. Saliencymaps [ 43]
canbeused tomodelhuman visionsystemand improve precisioninthese methods. Deeplearning forNR-IQA.Inrecentyears several works haveuseddeeplearningforNR-IQA[

1,11,12].Oneof

themaindra wbacksofdeep networksistheneedfor large labeleddatasets,which arecurrentlynot available forNR-

IQAresearch.T oaddressthis problemKangetal.[

11] considersmall32×32patchesratherthan images,thereby greatlyaugmen tingthenumberoftrainingexamples.The authorsof[

2,12]follow thesamepipeline.In[12]theau-

thorsdesigna multi-taskCNNto learnthetype ofdistor- tionsandimage qualitysimultaneously. Biancoatal. [ 1] proposetous eapre-trained networktomitigatethe lack oftrainingdata. Theye xtractfeaturesfrom apre-trained modelfine-tunedon anIQAdataset. Thesefeaturesare then usedtotrain anSVRmodel tomapfeatures toIQAscores. addressthelack oftrainingdata: weusea largenumber of automaticallygeneratedrankings ofimagequality totrain adeepnetw ork.Thisallo wsustotrainmuchdeeper and widernetworks thanothermethodsinNR-IQAwhich train directlyonabsolute IQAdata.

Learningtorank.Theseapproacheslearn arankingfunc-

tionfromground-truth rankingsbyminimizing aranking loss[

3].Thisfunction canthenbe appliedtorank testob-

jects.Theauthors of[

25]adaptthe StochasticGradientDe-

scentmethodto perfor mpairwiselearning torank.This hasbeensuccessfully appliedtolar gedatasets.Combining ideasfromranking andCNNs,the Siamesenetwork archi- tectureachiev esgreatsuccessonthefaceverification prob- lem[

5],andin comparingimagepatches [41].Theonly

otherwork whichappliesrankingsinthe contextofNR-quotesdbs_dbs29.pdfusesText_35

[PDF] source image mémoire

[PDF] bibliographie d'une image internet

[PDF] comment citer une source livre

[PDF] comment citer une loi française

[PDF] citer un article de loi en note de bas de page

[PDF] citer une loi norme apa

[PDF] citer article de loi apa

[PDF] citation ordre public droit administratif

[PDF] comment citer un texte de loi apa

[PDF] comment citer un article de la constitution

[PDF] citation secondaire

[PDF] normes apa dans le texte

[PDF] paraphraser online

[PDF] comment utiliser op-cit et ibid

[PDF] quand mettre un alinéa mémoire