[PDF] Density-Aware Single Image De-Raining Using a Multi-Stream

View - Download Density-Aware Single Image De-Raining Using a Multi-Stream

Previous PDF

Next PDF

Density-awareSingleImageDe-rainingusing aMulti-stream DenseNetwork

HeZhangV ishalM.P atel

DepartmentofElectrical andComputerEngineering

RutgersUniv ersity,Piscataway,NJ08854

{he.zhang92,vishal.m.patel}@rutgers.edu

Abstract

Singleimag erainstreakremovalis anextremelychal-

lengingproblem duetothepresenceof non-uniform raindensitiesinimag es.We presenta noveldensity- awaremulti-streamdenselyconnected convolutionalneural network-basedalgorithm,called DID-MDN,forjoint rain densityestimationand de-raining. Theproposed method enablesthenetwork itselftoautomatically determinethe rain-densityinformationandthenef ficientlyremo vethe correspondingrain-streaksguided bytheestimatedrain- densitylabel.T obetterc haracterizerain-streakswith differentscalesandshapes,amulti-str eamdenselycon- nectedde-raining networkisproposedwhich efficiently leveragesfeaturesfromdifferentscales.Furthermore ,a newdatasetcontainingimag eswithr ain-densitylabelsis createdandusedtotr ainthepr oposeddensity-aware net- work.Extensivee xperimentsonsynthetic andrealdatasets demonstratethattheproposed methodachie vessignifi- cantimpro vementsovertherecentstate-of-the-artmeth- ods.Inaddition, anablationstudy isperformedto demon- stratetheimprovements obtainedbydif ferentmodulesin theproposed method.Thecodecanbedownloaded at

1.Introduction

Inmany applicationssuchasdrone-basedvideosurv eil- lanceandself drivingcars, onehasto processimagesand videoscontainingundesirable artifactssuch asrain,sno w, andfog[ 8,7]or otherdistortionsuchasblur andlight [25].Furthermore,the performanceofman ycomputervi- sionsystemsoften degradeswhen theyare presentedwith imagescontainingsome oftheseartif acts.Hence,it isim- theseartifacts. Inthispaper,we addresstheproblem of rainstreakremo valfrom asingleimage.Variousmethods havebeenproposedintheliteratureto addressthisproblem [19,7,39, 21,3,16, 11,2,41, 36,6].

Oneofthe mainlimitationsof theexisting singleim-

(a) (b) (c) (d) (e) (f) Figure1:Imagede-rainingresults. (a)Inputrain yimage.(b) ResultfromFu etal.[7].(c) DID-MDN.(d)Input rainyimage. (e)Resultfrom Lietal.[36].(f) DID-MDN.Notethat [7]tendsto overde-raintheimagewhile[36]tends tounderde-rain theimage. agede-rainingmethods isthatthe yaredesigned todeal withcertaintypes ofrainy imagesandthe ydonot effec- tivelyconsidervariousshapes,scalesand densityofrain dropsintotheir algorithms.State-of-the-artde-raining al- gorithmssuchas [36,7]often tendtoo verde-rain orunder de-raintheimage iftherain conditionpresentin thetestim- ageisnot properlyconsideredduringtraining.Forexample, whenarain yimagesho wninFig.1(a)isde-rained usingthe methodofFu etal.[7],ittends toremov esomeimportant partsinthe de-rainedimagesuch astheright armofthe person,assho wninFig. 1(b).Similarly,when[36]is used tode-rainthe imageshown inFig.1(d), ittendsto under de-raintheimage andleav essomerain streaksinthe output de-rainedimage.He nce,moreadapti veandefficientmeth- ods,thatcan dealwithdif ferentraindensity levels present intheimage, areneeded. Onepossiblesolution tothisproblem istob uildav ery ingvarious rain-densitylevelswithdif ferentorientations andscales.This hasbeenachie vedby Fuetal.[7]and 1 695
Yangetal.[36],wherethe ysynthesizea novellarge-scale datasetconsistingof rainyimages withv ariousconditions andthey trainasinglenetworkbased onthisdataset forim- agede-raining.Ho wever ,onedrawbackofthisapproachis thatasingle networkmay notbecapable enoughtolearn alltypesof variationspresent inthetraining samples.Itcan beobserved fromFig.1thatbothmethods tendtoeither overde-rainorunderde-rainresults.Alternati vesolution tothisproblem istolearn adensity-specificmodel forde- raining.Howe ver,thissolutionlacksflexibilityinpractical de-rainingasthe densitylabelinformation isneededfor a givenrainyimagetodeterminewhich networktochoosefor de-raining.

Inorderto addresstheseissues, weproposea novel

streamDenseNetw ork(DID-MDN)that canautomatically determinetherain-density information(i.e.hea vy,medium orlight)present intheinput image(seeFig. 2).Thepro- posedmethodconsists oftwo mainstages:rain-density classificationandrain streakremov al.To accuratelyesti- matetherain-density level, anew residual-awareclassifier thatmakes useoftheresidualcomponent intherain yim- agefordens ityclassificationis proposedinthispaper.The rainstreakremo valalgorithm isbasedonamulti-stream densely-connectednetwork thattakesintoaccountthe dis- tinctscaleand shapeinformationof rainstreaks.Once therain-densityle velis estimated,wefusetheestimated densityinformationinto ourfinalmulti-stream densely- connectednetwork togetthefinalde-rainedoutput. Fur- thermore,toef ficientlytrainthe proposednetwork,alarge- scaledatasetconsisting of12,000im ageswithdif ferent rain-densitylev els/labels(i.e.heavy,mediumandlight) is synthesized.Fig.1(c) &(d)present sampleresultsfrom our network,whereonecanclearly seethatDID-MDN doesnot overde-rainorunderde-raintheimage andisable topro- videbetterresults ascomparedto [7]and[36].

Thispapermak esthefollo wingcontributions:

1.Ano velDID-MDNmethodwhichautomaticallydeter-

minestherain-density informationandthen efficiently estimatedrain-densitylabel isproposed.

2.Basedon theobservation thatresidualcan beusedas a

betterfeaturerepresentation incharacterizing therain- densityinformation,a novel residual-aware classifier toefficiently determinethedensity-levelof agiv en rainyimageisproposedin thispaper.

3.Ane wsyntheticdataset consistingof12,000training

imageswithrain-density labelsand1,200 testimages issynthesized.T othebest ofourknowledge,thisis thefirstdataset thatcontainsthe rain-densitylabelin- formation.Althoughthe networkis trainedonour syn- theticdataset,it generalizeswellto real-world rainy images.4.Extensiv eexperimentsareconductedonthreehighly challengingdatasets(tw osyntheticand onereal- world)andcomparisonsareperformed againstse veral recentstate-of-the-artapproaches. Furthermore,anab- lationstudyis conductedtodemonstrate theeffects of differentmodulesintheproposed network.

2.Background andRelatedWork

Inthissecti on,webriefly reviewseveral recentrelated worksonsingleimagede-raining andmulti-scalefeature aggregation.

2.1.SingleImage Deraining

Mathematically,arainyimage ycanbemodeled asalin-

earcombinationof arain-streakcomponent rwithaclean backgroundimagex,asfollo ws y=x+r.(1) Insingleimage de-raining,giv enythegoalis torecov er x.Ascan beobserved from(1)that imagede-rainingis ahighlyill-posed problem.Unlike vide o-basedmethods [26,32,28], whichlev eragetemporalinformation inre- movingraincomponents,prior-based methodshav ebeen proposedinthe literaturetodeal withthisproblem. These includesparsecoding-based methods[16,11, 47],low- rankrepresentation-basedmethods [3,39]and GMM-based (gaussianmixturemodel)methods[19]. Oneofthe limita- tionsofsome oftheseprior -basedmethodsis thatthey often tendtoo ver-smooth theimagedetails[16,39].

Recently,duetotheimmense successofdeep learning

inbothhigh-le veland low-levelvisiontasks[10,34,44,

23,35,37, 25],sev eralCNN-basedmethods have alsobeen

proposedforimage de-raining[4,6, 36,7].In thesemeth- ods,theidea istolearn amappingbetween inputrainy im- agesandtheir correspondinggroundtruths usingaCNN structure.

2.2.MultiscaleF eatureAggr egation

Ithasbeen observedthat combiningconv olutionalfea- turesatdif ferentlev els(scales)canleadtoa betterrep- resentationofan objectinthe imageandits surrounding context[9,45,13,38]. Forinstance, toefficiently lever - agefeaturesobtained fromdifferent scales,theFCN (fully andaddshigh-le velprediction layerstointermediatelayers togeneratepix el-wiseprediction resultsatmultipleresolu- tions.Similarly, theU-Netarchitecture[27]consistsof a contractingpathto capturetheconte xtanda symmetricex- pandingpaththat enablesthepreciselocalization.TheHED model[33]emplo ysdeeplysupervised structures,andau- tomaticallylearnsrich hierarchicalrepresentationsthat are fusedtoresolv ethechallenging ambiguityinedgeandob- jectboundarydetection. Multi-scalefeatures have alsobeen 696

Figure2:Anov erviewoftheproposedDID-MDNmethod.Theproposednetworkcontains twomodules: (a)residual-aw arerain-density

classifier,and(b)multi-streamdensely-connected de-rainingnetwork. Thegoalof theresidual-aw arerain-densityclassifier istodetermine

therain-densityle velgi venarainyimage.Ontheother hand,themulti-streamdensely-connectedde-rainingnetworkisdesigned to

efficientlyremovethe rainstreaksfromtherainyimagesguidedby theestimatedrain-density information. leveragedinvariousapplicationssuchas semanticsegmen- tation[45],f ace-alignment[22],visual tracking[18]crowd- counting[30],single imagesuper-resolution[43], faceanti- Spoofing[1],action recognition[48],depth estimation[5], singleimagedehazing [24,42,40] andalsoin singleim- agede-raining[36]. Similarto[36], wealsole veragea multi-streamnetwork tocapturetherain-streakcomponents withdifferent scalesandshapes.Howev er,rather thanus- ingtwo convolutionallayerswithdif ferentdilationfactors tocombinefeatures fromdifferent scales,wele veragethe densely-connectedblock[13] astheb uildingmoduleand thenweconnect featuresfromeach blocktogetherfor the finalrain-streakremo val.The ablationstudydemonstrates theeffecti venessofourproposednetworkcomparedwith thestructureproposed in[36].

3.Proposed Method

TheproposedDID-MDN architecturemainlyconsists

oftwo modules:(a)residual-awarerain-density classifier, and(b)multi-stream denselyconnectedde-raining network. Theresidual-aw arerain-densityclassifieraimstodetermine therain-densityle velgi venarainyimage.Ontheother hand,themulti-stream denselyconnectedde-raining net- workisdesignedtoef ficientlyremov etherain streaksfrom therainy imagesguidedbytheestimatedrai n-densityin- formation.Theentire networkarchitecture oftheproposed

DID-MDNmethodis shownin Fig.2.

3.1.Residualaware RaindensityClassifier

Asdiscussedabo ve,e venthoughsomeoftheprevi-

ousmethodsachie vesignificant improvementsonthede-rainingperformance,the yoftentend tooverde-rainor un-derde-rainthe image.Thisis mainlydueto thefact thatasinglenetw orkmaynot besufficientenoughtolearn dif-ferentrain-densitiesoccurring inpractice.W ebeliev ethatincorporatingdensitylevelinformationintothenetworkcanbenefittheo veralllearning procedureandhencecanguar-anteebettergeneralization todifferent rainconditions[26]. Similarobservations havealsobeenmade in[26],wheretheyusetwodif ferentpriorsto characterizelightrainandheavyrain,respectively .Unlike usingtwopriorstocharac-terizedifferent rain-densityconditions[26],therain-densitylabelestimatedfromaCNNclassifierisusedforguidingthede-rainingprocess.T oaccuratelyestimate thedensityin-formationgiv enarainyinputimage,aresidual-aw arerain-densityclassifieris proposed,wherethe residualinforma-tionisle veragedto betterrepresenttherainfeatures.Inaddition,totrain theclassier, alarge-scale syntheticdatasetconsistingof12,000 rainyimages withdensitylabels issyn-thesized.Notethat thereareonly threetypesof classes(i.e.labels)presentin thedatasetand theycorrespond tolow ,mediumandhigh density.

Onecommonstrate gyintraining anewclassifieristo

ofthefundamental reasonstole veragea fine-tunestrate gy forthene wdatasetis thatdiscriminativefeaturesencoded inthesepre-defined modelscanbe beneficialinaccelerat- ingthetraining anditcan alsoguaranteebetter generaliza- tion.Howe ver,weobservedthatdirectlyfine-tuningsucha 'deep"modelon ourtaskis notanef ficientsolution. Thisis mainlydueto thefact thathigh-lev elfeatures(deeper part) 697

ofaCNN tendtopay moreattentionto localizethediscrim- inativeobjectsintheinputimage[46]. Hence,relativ elysmallrain-streaksmay notbelocalized wellinthese high-levelfeatures.Inotherwords,the rain-streakinformationmaybelost inthehigh-le velfeatures andhencemay de-gradetheo verallclassification performance.Asaresult,itisimportantto comeupwith abetterfeature representationtoeffecti velycharacterizerain-streaks(i.e.rain-density).

From(1),one canreg ardr=y-xastheresidual com-

ponentwhichcan beusedto characterizetherain-density . Toestimatethe residualcomponent( ˆr)fromthe observa- tiony,amulti-stream dense-net(withoutthe labelfusion part)usingthe newdataset withheavy-density istrained. Then,theestimated residualisre gardedas theinputto trainthefinal classifier.In thisway ,theresidualestimation partcanbe regarded asthefeature extractionprocedure, 1 whichisdiscussed inSection3.2. Theclassificationpart ismainlycomposed ofthreecon volutionallayers (Conv) withkernel size3×3,onea veragepooli ng(AP)layer withkernel size9×9andtw ofully-connectedla yers(FC).

Detailsofthe classifierareas follows:

Conv(3,24)-Conv(24,64)-Conv(64,24)-AP-

FC(127896,512)-FC(512,3),

where(3,24)means thattheinput consistsof3 channelsand theoutputconsists of24channels. Notethatthe finallayer consistsofa setof3 neuronsindicatingthe rain-density classofthe inputimage(i.e. low, medium,high).An ablationstudy, discussedinSection4.3,isconducted to demonstratetheef fectiveness ofproposedresidual-aware classifierascompared withtheV GG-16[29]model. Lossfor theResidual-awareClassifier:.To efficiently traintheclassifier ,atw o-stagetrainingprotocolislev er- aged.Aresidual featureextraction networkis firstlytrained toestimate theresidualpartofthegi venrain yimage,then aclassificationsub-netw orkistrained usingtheestimated residualasthe inputandis optimizedviathe groundtruth labels(rain-density).Finally ,thetw ostages(featureextrac- tionandclassification) arejointlyoptimized. Theov erall lossfunctionused totrainthe residual-aware classierisas follows:

L=LE,r+LC,(2)

whereLE,rindicatestheper -pixelEuclidean-loss toesti- matetheresidual componentandLCindicatesthecross-

3.2.Multistream DenseNetwork

Itiswell-kno wnthatdif ferentrainyimagescontainrain- streakswithdif ferentscalesand shapes.Consideringthe imagesshown inFig.3,therainy imageinFig. 3(a)con- tainssmallerrain-streaks, whichcanbe capturedbysmall-

1Classificaitonnetwork canberegardedas twoparts: 1.Featureextrac-

torand2. Classifer (a) (b) Figure3:Sampleimagescontaining rain-streakswithv arious scalesandshapes .(a)containssmaller rain-streaks,(b)contains longerrain-streaks. scalefeatures(with smallerreceptiv efields),while theim- ageinFig. 3(b)contains longerrain-streaks,which can becapturedby large-scalefeatures (withlarger receptive fields).Hence,we believe thatcombiningfeatures fromdif- ferentscalescan beamore efficientw aytocapture various rainstreakcomponents [12,36].

Basedonthis observationand motivated bythesuccess

ofusingmulti-scale featuresforsingle imagede-raining [36],amore efficientmulti-stream densely-conne ctednet- worktoestimatetherain-streak componentsisproposed, whereeachstream isbuilt onthedense-block introducedin [13]withdif ferentkernel sizes(differentreceptive fields). Thesemulti-streamblocks aredenotedby Dense1(7×7), Dense2(5×5),andDense3 (3×3),inyello w,green and blueblocks,respecti velyin Fig.2.Inaddition,tofurther improvetheinformationflowamongdif ferentblocksand tolev eragefeaturesfromeachdense-blockinestimating therainstreak components,amodified connectivityis intro- duced,whereall thefeaturesfrom eachblockare concate- natedtogetherfor rain-streakestimation.Rather thanlev er- agingonlytw oconv olutionallayersin eachstream[36],we createshortpaths amongfeaturesfrom differentscales to strengthenfeatureaggre gationand toobtainbetterconver- gence.To demonstratetheeffectiveness ofourproposed multi-streamnetwork comparedwiththemulti-scalestruc- tureproposedin [36],anablat ionstudyis conducted,which isdescribedin Section4. Toleveragethe rain-densityinformationtoguidethede- rainingprocess,the up-sampledlabelmap

2isconcatenated

withtherain streakfeaturesfrom allthreestreams. Then, theconcatenatedfeatures areusedto estimatetheresidual (ˆr)rain-streakinformation. Inaddition,the residualissub- tractedfromthe inputrainy imagetoestimate thecoarse de-rainedimage.Finally ,tofurther refinetheestimated coarsede-rainedimage andmake surebetterdetails well preserved,anothertwocon volutionallayers withReLUare

2Forexample,ifthe labelis1,thenthecorresponding up-sampled

label-mapisof thesamedimension astheoutput featuresfromeach stream andallthe pixelv aluesofthe labelmapare1. 698
adoptedasthe finalrefinement.

Therearesix dense-blocksineach stream.Mathemati-

cally,eachstreamcanbe representedas s j=cat[DB1,DB2,...,DB6],(3) notestheoutput fromtheithdenseblock, andsj,j=1,2,3 denotesthejthstream.Furthermore, weadoptdif ferent transitionlayercombinations

3andkernel sizesineach

stream.Detailsof eachstreamare asfollows: Dense1:threetransiti on-d ownlayers,threetransition-up layersandk ernelsize7×7.

Dense2:two transition-downlayers,twono-sampling

transitionlayers,tw otransition-uplayers andkernelsize

5×5.

Dense3:onetransition-do wnlayer, fourno-sampling

transitionlayers,one transition-uplayerand kernelsize

3×3.

Notethateach dense-blockisfollo wedbya transitionlayer. Fig4presents anov erviewof thefirststream, Dense1.

Figure4:Detailsofthe firststreamDense1.

Lossfor theDe-rainingNetwork:.Motiv atedbytheob-

servationthatCNNfeature-based losscanbetter improve thesemantic edgeinformation[15,17] andtofurther en- hancethevisual qualityofthe estimatedde-rainedimage [41],wealso leverage aweightedcombination ofpixel- wiseEuclideanloss andthefeature-based loss.Theloss fortrainingthe multi-streamdensely connectednetwork is asfollows

L=LE,r+LE,d+λFLF,(4)

whereLE,drepresentstheper -pixelEuclidean lossfunction toreconstructthe de-rainedimageand LFisthefeature- basedlossfor thede-rainedimage, definedas L F=1

CWH?F(ˆx)c,w,h-F(x)c,w,h?22,(5)

whereFrepresentsanon-linear CNNtransformationand ˆx isthereco veredde-rained image.Here,wehaveassumed thatthefeatures areofsize w×hwithcchannels.Inour method,wecompute thefeatureloss fromthelayer relu1 2 oftheV GG-16model[29].

3Thetransitionlayer canfunctionas up-sampletransition,do wn-

sampletransitionor no-samplingtransition[14]. 3.3.Testing Duringtesting,the rain-densitylabel informationusing theproposedresidual-a wareclassifier isestimated.Then, theup-sampledlabel-map withthecorresponding inputim- agearefed intothemulti-stream networkto getthefinal de-rainedimage.

4.ExperimentalResults

Inthissection, wepresentthe experimentaldetails and evaluationresultsonbothsyntheticandreal-w orlddatasets. De-rainingperformanceon thesyntheticdat aise valuated intermsof PSNRandSSIM [31].Performanceof different methodsonreal-w orldimagesis evaluatedvisuallysince thegroundtruth imagesare notav ailable.Theproposed

DID-MDNmethodis comparedwiththe followingrecent

state-of-the-artmethods:(a) Discriminative sparsecoding- basedmethod(DSC) [21](ICCV"15),(b) Gaussianmixture model(GMM)based method[19] (CVPR"16),(c)CNN method(CNN)[6] (TIP"17),(d)Joint RainDetectionand

Removal(JORDER)method[36](CVPR"17),(e)Deep de-

tailedNetwork method(DDN)[7](CVPR"17),and(f) Joint

Bi-layerOptimization(JBO) method[47](ICCV"17).

4.1.SyntheticDataset

Eventhoughthereexist several large-scalesynthetic

datasets[7,41, 36],they lackthea vailabilityof thecorre- spondingrain-densitylabel informationforeach synthetic rainyimage.Hence,wede velopa newdataset, denoted asTrain1,consistingof 12,000images,where eachimage isassigneda labelbasedon itscorrespondingrain-density level.Therearethreerain-densitylabelspresent inthe dataset(e.g.light, mediumandhea vy).Thereare roughly

4,000imagesper rain-densitylev elinthe dataset.Similarly,

wealsosynthesize anew testset,denoted asTest1,which consistsofa totalof1,200 images.Itis ensuredthateach datasetcontainsrain streakswithdif ferentorientationsand scales.Imagesare synthesizedusingPhotoshop. Wemod- ifythenoise level introducedinstep 3of

4togeneratedif-

ferentrain-densityimages, wherelight,medium andheavy rainconditionscorrespond tothenoise levels 5%≂35%,

35%≂65%,and65%≂95%,respectiv ely5.Samplesyn-

thesizedimagesunder thesethreeconditions aresho wnin Fig5.T obettertest thegeneralizationcapabilityofthepro- posedmethod, wealsorandomlysample1,000images from thesyntheticdataset providedby Fu[7]as anothertesting set,denotedas Test2. effects/photoshopweather-effects-rain/

5Thereasonwh yweuse threelabelsisthatduringour experiments,we

foundthatha vingmorethan threerain-densitylevelsdoes notsignificantly improvetheperformance.Hence,weonlyuse threelabels(hea vy,medium andlight)in theexperiments. 699
Table1:Quantitativeresultsevaluatedinterms ofav erageSSIMandPSNR(dB)(SSIM/PSNR).

InputDSC[21](ICCV"15) GMM[19](CVPR"16) CNN[6](TIP"17) JORDER[36](CVPR"17) DDN[7](CVPR"17) JBO[47](ICCV"17) DID-MDN

Heavy

Medium

Light Figure5:Samplessyntheticimages inthreedif ferentconditions. urationsonTest1.

PSNR(dB)26.0526.7527.5627.95

SSIM0.88930.89010.90280.9087

Table3:Accuracyofrain-densityestimatione valuatedon Test1.

VGG-16[29]Residual-aware

Accuracy73.32%85.15%

4.2.Training Details

Duringtraining,a 512×512imageis randomly

croppedfromthe inputimage(or itshorizontalflip) ofsize

586×586.Adamis usedasoptimization algorithmwith a

mini-batchsizeof 1.Thelearning ratestartsfrom 0.001 andisdi videdby10 after20epoch.Themodelsare trained forupto 80×12000iterations.W eusea weightdecay of0.0001and amomentumof 0.9.Theentire network istrainedusing thePytorchframe work.During training, wesetλF=1.Allthe parametersaredefined viacross- validationusingthevalidation set.

4.3.AblationStudy

Thefirstablation studyisconducted todemonstratethe paredtothe VGG-16[29] model.Thetw oclassifiersare trainedusingour synthesizedtrainingsamples Train1and testedonthe Test1set.Theclassification accuracycorre- spondingtoboth classifiersonTest1istabulated inTable3. Itcanbe observedthat theproposedresidual-a wareclassi- fierismore accuratethanthe VGG-16model forpredicting therain-densityle vels. Inthesecond ablationstudy, wedemonstratethe effec- tivenessofdifferentmodulesinour methodby conducting thefollowing experiments: •Single:Asingle-stream denselyconnectednetw ork (Dense2)withoutthe procedureoflabel fusion.•Yang-Multi[36]6:Multi-streamnetw orktrained withouttheprocedure oflabelfusion. •Multi-no-label:Multi-streamdensely connectednet- worktrainedwithouttheprocedure oflabelfusion. networktrainedwiththeprocedure ofestimatedlabel fusion.

Theav eragePSNRandSSIMresultsevaluatedon Test1

aretabulated inTable2.Assho wninFig. 6,eventhough thesinglestream networkand Yang"s multi-streamnetwork [36]areable tosuccessfullyremo vethe rainstreakcom- ponents,they bothtendtoover de-raintheimage withthe blurryoutput.The multi-streamnetwork withoutlabelfu- sionisunable toaccuratelyestimate therain-densityle vel andhenceit tendstolea vesome rainstreaks inthede- rainedimage(especially observedfrom thederained-part aroundthelight). Incontrast,the proposedmulti-stream networkwithlabelfusionapproach iscapableof removing rainstreakswhile preservingthebackground details.Sim- ilarobservations canbemadeusingthequantitati veresults asshown inTable2.

4.3.1Resultson Two SyntheticDatasets

Wecomparequantitative andqualitativ eperformanceof differentmethodsonthetest imagesfromthe twosynthetic datasets-Test1andTest2.Quantitativ eresultscorrespond- ingtodif ferentmethodsare tabulatedinTable1. Itcan beclearlyobserv edthatthe proposedDID-MDNisableto achievesuperiorquantitativeperformance.

Tovisuallydemonstratetheimpro vementsobtained by

theproposedmethod onthesynthetic dataset,resultson two sampleimagesselected fromTest2andonesample chosen fromourne wlysynthesizedTest1arepresentedin Figure7. Notethatwe selectively sampleimagesfrom allthreecon- ditionstosho wthatour methodperformswellunderdiffer - entvariations

7.Whilethe JORDERmethod[36] isable

toremov esomepartsoftherain-streaks,itstilltends to leavesomerain-streaksinthede-rainedimages. Similarre- sultsarealso observedfrom [47].Even thoughthemethod

6Tobetterdemonstratetheef fectiveness ofourproposed muli-stream

networkcomparedwiththestate-of-the-art multi-scalestructureproposed in[36],we replaceourmulti-stream dense-netpartwith themulti-scale structuredin[36] andkeep alltheother partsthesame.

7Duetospace limitationsandfor bettercomparisons,we onlyshow the

resultscorrespondingto themostrecent state-of-the-artmethods[36, 7, 47].
700

PSNR:16.47SSIM:0.51

InputPSNR:22.87SSIM:0.8215

SinglePSNR:23.02SSIM:0.8213

Yang-Multi[36]PSNR:23.47SSIM:0.8233

Multi-no-labelPSNR:24.88

SSIM:0.8623

quotesdbs_dbs47.pdfusesText_47

[PDF] multiples et diviseurs 5ème

[PDF] multiples et diviseurs 5ème exercices

[PDF] multiples et diviseurs 6ème

[PDF] multiples et diviseurs cm1

[PDF] multiples et diviseurs cm2

[PDF] multiples et diviseurs de 4

[PDF] multiples et diviseurs exercices

[PDF] multiples et diviseurs exercices ? imprimer

[PDF] Multiples et pourcentages de mathematiques

[PDF] Multiples et sous multiples de l'intensité

[PDF] multiplicateur de budget équilibré

[PDF] multiplicateur fiscal calcul

[PDF] multiplication

[PDF] Multiplication

[PDF] multiplication 1-12 worksheets