[PDF] mit starting salary by major
[PDF] mitratel
[PDF] mixed expressive receptive language processing disorder
[PDF] mixtures and solutions lab activities
[PDF] mixtures and solutions module investigation 3 concentration
[PDF] mixtures and solutions website
[PDF] mixtures and solutions worksheet
[PDF] ml engineering
[PDF] ml tests
[PDF] mobile app business plan doc
[PDF] mobile app design plan template
[PDF] mobile app development project plan
[PDF] mobile app development project plan template
[PDF] mobile app development proposal ppt
[PDF] mobile app documentation template
Lecture ffi
Artificial
Neural Networks
w L L y oitewnepr vIhipr vItdaifk xf orj, f.om vm gy.ijηm cf .imyijmv .δ orm m .fmgo, orco rcim mη o. orm mmymfm .δ yojjc myc mov.y, c, c c.y gcycηj δ.y coc jfjf cggjcoj.f, myc fmo, rcim .fm ory.r ov. c.y ηmim.gmfo gmyj.η, orm mcy !", cfη orm jη #!", $rm vmym c m ηmim.gmfo jf orm mη .δ crjfm mcyfjf yojjc myc mov.y, vmym jf,gjymη % %j..jc fηjf, ymcojf o. orm %mrcij.y .δ orm %ycjf c, c fmov.y .δ fjo, cmη fmy.f, $rm rcf %ycjf j, m,ojcomη o. rcim cy.fη w! %jj.f fmy.f, mcr .ffmomη .f cimycm o. w!&!!! .ormy fmy.f, 'cr fmy.f ymmjim, ,jfc, ory.r ,fcg,m, orco .foy. orm m(mo, .δ orm ,jfc .f orm fmy.f $rm,m ,fcgoj .ffmoj.f, cym %mjmimη o. gc c m y.m jf orm %mrcij.y .δ orm %ycjf $rm δfηcmfoc %jηjf %. jf cf yojjc myc mov.y j, orm cormcojc .ηm .δ c fmy.f c, ,r.vf jf )jym w $rm orymm %c,j .g.fmfo, .δ orm *cyojjc+ fmy.f cym, w $rm ,fcg,m, .y .ffmojf jf, orco gy.ijηm vmjro,& - F o. orm jfgo icm,& } F
± -w.
f cηηmy orco ,, orm vmjromη jfgo icm, o. .gom orm i jfgo o. orm cojicoj.f δfoj.f 0 F F vrmym j, cmη orm F?? %jc, *f.o o. %m .fδ,mη vjor ,ocoj,ojc %jc, jf gymηjoj.f .y m,ojcoj.f+ j, c fmyjc icm c,,.jcomη vjor orm fmy.f xo j, .fimfjmfo o. orjf .δ orm %jc, c, orm vmjro δ.y cf jfgo } vr.,m icm j, cvc, m1c o. .fm& i orco F F F?? 2 f cojicoj.f δfoj.f *c,. cmη c ,1c,rjf δfoj.f+ orco cg, o. * + orm .ogo icm .δ orm fmy.f $rj, δfoj.f j, c .f.o.fm
δfoj.f
Lectur 8
While there are numerous different (artificial) neural network architec tures that have been studied by researchers, the most successful applica tions in data mining of neural networks have been multilayer feedforward networks. These are networks in which there is an input layer consisting of nodes that simply accept the input values and successive layers of nodes that are neurons as depicted in Figure 1. The outputs of neurons in a layer are inputs to neurons in the next layer. The last layer is called the output layer. Layers between the input and output layers are known as hidden layers. Figure 2 is a diagram for this architecture.
Lectur
A In a supervised setting where a neural net is used to predict a numerical quantity there is one neuron in the output layer and its output is the predic tion. When the network is used for classification, the output layer typically has as many nodes as the number of classes and the output layer node with 3 L the largest output value gives the network's estimate of the class for a given input. In the special case of two classes it is common to have just one node in the output layer, the classification between the two classes being made by applying a cut-off to the output value at the node. myx ygxj xoηcjδ Let us begin by examining neural networks with just one layer of neurons (output layer only, no hidden layers). The simplest network consists of just one neuron with the function chosen to be the identity function, ( for all . In this case notice that the output of the network is i F F ,a F?? linear function of the input vector } with components } F
If we are modeling
the dependent variable using multiple linear regression, we can interpret the neural network as a structure that predicts a value e for a given input vector } with the weights being the coefficients. If we choose these weights to minimize the mean square error using observations in a training set, these weights would simply be the least squares estimates of the coefficients. The weights in neural nets are also often designed to minimize mean square error in a training data set. There is, however, a different orientation in the case of neural nets: the weights are "learned". The network is presented with cases from the training data one at a time and the weights are revised after each case in an attempt to minimize the mean square error. This process of incremental adjustment of weights is based on the error made on training cases and is known as 'training' the neural net. The almost universally used dynamic updating algorithm for the neural net version of linear regression is known as the Widrow-Hoff rule or the least-mean-square (LMS) algorithm. It is simply stated. Let }( ) denote the input vector } for the gu case used to train the network, and the weights Lectue this case is presented to the net by the vector -( ) The updating rule is -( +1) = -( e( ) with -(0) = 0. It can be shown that if the network is trained in this manner by repeatedly presenting test data observations one-at-a-time then for suitably small (absolute) values of the network will learn (converge to) the optimal values of -. Note that the training data may have to be presented several times for -( ) to be close to the optimal -. The advantage of dynamic updating is that the network tracks moderate time trends in the underlying linear model quite effectively. If we consider using the single layer neural net for classification into c classes, we would use c nodes in the output layer. If we think of classical 4 t discriminant analysis in neural network terms, the coefficients in Fisher's classification functions give us weights for the network that are optimal if the input vectors come from Multivariate Normal distributions with a common covariance matrix. For classification into two classes, the linear optimization approach that we examined in class, can be viewed as choosing optimal weights in a single layer neural network using the appropriate objective function. Maximum likelihood coefficients for logistic regression can also be con sidered as weights in a neural network to minimize a function of c the residuals r called the deviance. In this case the logistic function O(s)= ??r is the activation function for the output node. 1.2
Multilayer Neural networks
Multilayer neural networks are undoubtedly the most popular networks used in applications. While it is possible to consider many activation functions, in practice c it has t been found that the logistic (also called the sigmoid) function r O(s)= ??r as the activation function (or minor variants such as the tanh function) works best. In fact the revival of interest in neural nets was sparked by successes in training neural networks using this function in place of the historically (biologically inspired) step function (the "perceptron"}.
Notice
that using a linear function does not achieve anything in multilayer networks that is beyond what can be done with single layer networks with linear activation functions. The practical value of the logistic function arises from the fact that it is almost linear in the range where g is between 0.1 and 0.9 but has a squashing effect on very small or very large values of s. In theory it is sufficient to consider networks with two layers of neurons- one hidden and one output layer-and this is certainly the case for most applications. There are, however, a number of situations where three and sometimes four and five layers have been more effective. For prediction the output node is often given a linear activation function to provide forecasts that are not limited to the zero to one range. An alternative is to scale the output to the linear part (0.1 to 0.9) of the logistic function.
Unfortunately
there is no clear theory to guide us on choosing the number of nodes in each hidden layer or indeed the number of layers. The common practice is to use trial and error, although there are schemes for combining 5 Aatrnr7otrAc nituAw8 83.u o8 gicitr. oSgAlrtun8 firtu citfiAls tlorcrcg kAl tui8i aolonitil8p :rc.i tlroS ocw illAl r8 o ci.i88ol1 aolt Ak ci3loS cit oaaSr.otrAc8 rt r8 rnaAltoct tA uoNi oc 3cwil8tocwrcg Ak tui 8tocwolw nituAw 38iw tA tlorc o n3StrSo1iliw citfiAls6 4o.salAaogotrAcp et r8 cA i4oggilotrAc tA 8o1 tuot tui
8aiiw Ak tui 4o.salAa oSgAlrtun nowi ci3loS cit8 o alo.tr.oS tAAS rc
tui noccil tuot tui 8rnaSi4 nituAw nowi Srciol Aatrnr7otrAc o alo.tr.oS tAASp dui liNrNoS Ak 8tlAcg rctili8t rc ci3loS cit8 rc tui nrw Ih8 fio8 rc Solgi nio83li w3i tA tui i6.ric.1 Ak tui 4o.salAa oSgAlrtunp ycu jDpM.rIy- 6ek0Ii'k 8iek Tptp 5it
38 SAAs ot tui elr8 woto tuot 5r8uil ocoS17iw 38rcg xr8.lrnrcoct ycoS18r8p
@i.oSS tuot tui woto .Ac8r8tiw Ak kA3l nio83linict8 Ac tulii t1ai8 Ak rlr8 AAfiil8p duili oli ?h A48ilNotrAc8 kAl io.u .So88 Ak rlr8p y aolt Ak tui woto r8 lialAw3.iw 4iSAfip F F BC:D :EBFeB: F5y::FBxB :BE5BM :BE3 EBd5BM EBd3 L elr8:8itA8o L ?pL 2p? Lp; hp/ elr8:8itA8o L ;p< 2 Lp; hp/ 2 elr8:8itA8o L ;pG 2p/ Lp2 hp/ elr8:8itA8o L ;pF2pL Lp? hp/ elr8:8itA8o L ? 2pFLp; hp/
Felr8:8itA8o
L ?p; 2p< LpG hp;
G elr8:8itA8o L ;pF2p; Lp; hp2 I elr8:8itA8o L ? 2p; Lp? hp/ elr8:8itA8o L ;p; /p< Lp; hp/ Lh elr8:8itA8o L ;p< 2pL Lp? hpL ppp ppp ppp ppp ppp ppp ppp ?L elr8:Nil8r.ASAl / G 2p/ ;pG Lp; elr8:Nil8r.ASAl / Fp; 2p/ ;p? Lp? ?2 elr8:Nil8r.ASAl / Fp< 2pL ;p< Lp? elr8:Nil8r.ASAl / ?p? /p2 ; Lp2 elr8:Nil8r.ASAl / Fp? /pI ;pF Lp? ?Felr8:Nil8r.ASAl / ?pG /pI ;p? Lp2 ?G elr8:Nil8r.ASAl / Fp2 2p2 ;pG LpF ?I elr8:Nil8r.ASAl / ;p< /p; 2p2 L elr8:Nil8r.ASAl / FpF /p< ;pF Lp2 Fh elr8:Nil8r.ASAl / ?p/ /pG 2p< Lp; ppp ppp ppp ppp ppp ppp ppp LhL elr8:Nrlgrcr.o 2 Fp2 2p2 F /p? Lh/ elr8:Nrlgrcr.o 2 ?pI /pG ?pL Lp< Lh2 elr8:Nrlgrcr.o 2 GpL 2 ?p< /pL Lh; elr8:Nrlgrcr.o 2 Fp2 /p< ?pF LpI Lh? elr8:Nrlgrcr.o 2 Fp? 2 ?pI /p/
LhFelr8:Nrlgrcr.o
2 GpF 2 pF /pL
LhG elr8:Nrlgrcr.o 2 ;p< /p? ;p? LpG LhI elr8:Nrlgrcr.o 2 Gp2 /p< Fp2 LpI Lh< elr8:Nrlgrcr.o 2 FpG /p? ?pI LpI LLh elr8:Nrlgrcr.o 2 Gp/ 2pFFpL /p? ek fii 38i o ci3loS cit ol.urti.t3li kAl tur8 .So88r2.otrAc alA4Sin fii firSS ciiw ; cAwi8 vcAt .A3ctrcg tui 4ro8 cAwif Aci kAl io.u Ak tui ; rcwiaicwict Nolro4Si8 rc tui rca3t So1il ocw 2 ci3lAc8 vAci kAl io.u .So88f rc tui A3ta3t So1ilp 5it 38 8iSi.t Aci urwwic So1il firtu /? ci3lAc8p MAtr.i tuot tuili firSS
4i o tAtoS Ak /? .Acci.trAc8 klAn io.u cAwi rc tui rca3t So1il tA cAwi8 rc
tui urwwic So1ilp dur8 nosi8 o tAtoS Ak;4/?RLhh .Acci.trAc8 4itfiiic G tui rca3t So1il ocw tui urwwic So1ilp ec owwrtrAc tuili firSS 4i o tAtoS Ak 2 .Acci.trAc8 klAn io.u cAwi rc tui urwwic So1il tA cAwi8 rc tui A3ta3t So1ilp dur8 nosi8 o tAtoS Ak /?42RG? .Acci.trAc8 4itfiiic tui urwwic So1il ocw tui A3ta3t So1ilp >8rcg tui 8tocwolw SAgr8tr. o.trNotrAc k3c.trAc8T tui cit: fiAls fio8 tlorciw firtu o l3c .Ac8r8trcg Ak FhThhh rtilotrAc8p Bo.u rtilotrAc .Ac8r8t8 Ak ali8ictotrAc tA tui rca3t So1il Ak tui rcwiaicwict Nolro4Si8 rc o .o8iT kASSAfiiw 41 83..i88rNi .Ana3totrAc8 Ak tui A3ta3t8 Ak tui ci3lAc8 Ak tui urwwic So1il ocw tui A3ta3t So1il 38rcg tui oaalAalroti fiirgut8p dui
A3ta3t
NoS3i8 Ak ci3lAc8 rc tui A3ta3t So1il oli 38iw tA .Ana3ti tui ilQ lAlp dur8 illAl r8 38iw tA owE38t tui fiirgut8 Ak oSS tui .Acci.trAc8 rc tui citfiAls 38rcg tui 4o.sfiolw alAaogotrAc vH4o.salAa8f tA .AnaSiti tui rtQ ilotrAcp :rc.i tui tlorcrcg woto uo8 L?h .o8i8T io.u .o8i fio8 ali8ictiw tA tui citfiAls ;hh trni8p ycAtuil fio1 Ak 8totrcg tur8 r8 tA 8o1 tui citfiAls fio8 tlorciw kAl ;hh iaA.u8 fiuili oc iaA.u .Ac8r8t8 Ak Aci 8fiiia tulA3gu tui ictrli tlorcrcg wotop dui li83St8 kAl tui So8t iaA.u Ak tlorcrcg tui ci3loS cit
Ac tur8 woto oli 8uAfic 4iSAfi6
ehIw ptrvtr s dafwwIkyfrIcl dclmtwIcl gfrhIA xi8rliw
FAna3tiw FSo88
FSo88
L / 2 dAtoS
L ?h ?h ;< L?h 2
L ; dAtoS ?h ?h ?h L?h fihhch Nivchr
FSo88 Eottilc8 D BllAl8 I BllAl8 :twxiN
L ?h h hphh v hphhf ?h L /phh v LpBNiloSS L?h / Lp2 v hp dui .So88r2.otrAc illAl Ak Lp2I r8 4ittil tuoc tui illAl 38rcg wr8.lrnrcoct ocoS18r8 fiur.u fio8 /I v:ii Si.t3li cAti Ac xr8.lrnrcoct ycoS18r8fp MAtr.i tuot uow fii 8tAaaiw oktil AcS1 Aci ao88 Ak tui woto vL?h rtilotrAc8f tui I illAl r8 n3.u fiAl8i vG?If o8 8uAfic 4iSAfi6 ehIw ptrvtr o dafwwIkyfrIcl dclmtwIcl gfrhIA xi8rliw FAna3tiw FSo88
FSo88 L / 2 dAtoS
L LhG / L<
L2L F /h
2 L/? ; /L
dAtoS 2? L2 L/ Fh dui .So88r2.otrAc illAl loti Ak Lp2I fio8 A4torciw 41 .olik3S .uAr.i Ak si1 .ActlAS aolonitil8 kAl tui tlorcrcg l3c 41 tlroS ocw illAlp ek fii 8it tui .ActlAS aolonitil8 tA aAAl NoS3i8 fii .oc uoNi tillr4Si li83St8p dA 3cQ wil8tocw tui aolonitil8 rcNASNiw fii ciiw tA 3cwil8tocw uAfi tui 4o.sfiolw alAaogotrAc oSgAlrtun fiAls8p &!x 'gfδηgj$ (jcggomc .ycjmo! 3i firSS wr8.388 tui 4o.salAa oSgAlrtun kAl .So88r2.otrAc alA4Sin8p duili r8
o nrcAl owE38tnict kAl aliwr.trAc alA4Sin8 fiuili fii oli tl1rcg tA aliwr.t o .Actrc3A38 c3nilr.oS NoS3ip ec tuot 8rt3otrAc fii .uocgi tui o.trNotrAc k3c.trAc kAl A3ta3t So1il ci3lAc8 tA tui rwictrt1 k3c.trAc tuot uo8 }R }p vyc oStilcotrNi r8 tA li8.oSi ocw li.ictil tui SAgr8tr. k3c.trAc tA ailnrt tui A3ta3t8 tA 4i oaalA4rnotiS1 Srciol rc tui locgi Ak wiaicwict Nolro4Si NoS3i8fp dui 4o.salAa oSgAlrtun .1.Si8 tulA3gu tfiA wr8trc.t ao88i8T o kAlfiolw
ao88 kASSAfiiw 41 o 4o.sfiolw ao88 tulA3gu tui So1il8 Ak tui citfiAlsp dui oSgAlrtun oStilcoti8 4itfiiic tui8i ao88i8 8iNiloS trni8 o8 rt 8.oc8 tui tlorcQ rcg wotop d1ar.oSS1T tui tlorcrcg woto uo8 tA 4i 8.occiw 8iNiloS trni8 4ikAli tui citfiAls8 8Siolc88 tA nosi gAAw .So88r2.otrAc8p 1chufhj DfwwM dc.vtrfrIcl cm ctrvtrw cm faa r-i lithclw Il
r-i liruch6 dui oSgAlrtun 8tolt8 firtu tui 2l8t urwwic So1il 38rcg o8 rca3t NoS3i8 tui rcwiaicwict Nolro4Si8 Ak o .o8i vAktic .oSSiw oc i4inaSol rc tui no.urci Siolcrcg .Ann3crt1f klAn tui tlorcrcg woto 8itp dui ci3lAc A3ta3t8
oli .Ana3tiw kAl oSS ci3lAc8 rc tui 2l8t urwwic So1il 41 ailkAlnrcg tui liSiNoct 83n ocw o.trNotrAc k3c.trAc iNoS3otrAc8p dui8i A3ta3t8 oli tui rca3t8 kAl ci3lAc8 rc tui 8i.Acw urwwic So1ilp ygorc tui liSiNoct 83n ocw o.trNotrAc k3c.trAc .oS.3SotrAc8 oli ailkAlniw tA .Ana3ti tui A3ta3t8 Ak 8i.Acw So1il ci3lAc8p dur8 .Actrc3i8 So1il 41 So1il 3ctrS fii lio.u tui
A3ta3t
So1il ocw .Ana3ti tui A3ta3t8 kAl tur8 So1ilp dui8i A3ta3t NoS3i8 .Ac8trt3ti tui ci3loS cit"8 g3i88 ot tui NoS3i Ak tui wiaicwict Nolro4Sip ek fii oli 38rcg tui ci3loS cit kAl .So88r2.otrAcT ocw fii uoNi . .So88i8T fii firSS uoNi . ci3lAc A3ta3t8 klAn tui o.trNotrAc k3c.trAc8 ocw fii 38i tui Solgi8t NoS3i tA witilnrci tui cit"8 .So88r2.otrAcp vek a R /T fii .oc 38i E38t Aci A3ta3t
cAwi firtu o .3t:AP NoS3i tA noa oc c3nilr.oS A3ta3t NoS3i tA Aci Ak tui tfiA .So88i8fp 5it 38 wicAti 41 1
jw tui fiirgut Ak tui .Acci.trAc klAn cAwi r tA cAwi Ep dui NoS3i8 Ak 1
jw oli rcrtroSr7iw tA 8noSS vgiciloSS1 locwAnf c3n4il8 rc tui locgi hohh ± hoh?p dui8i fiirgut8 oli owE38tiw tA cifi NoS3i8 rc tui 4o.sfiolw ao88 o8 wi8.lr4iw 4iSAfip Backward pass: Propagation of error and adjustment of weights dur8 auo8i 4igrc8 firtu tui .Ana3totrAc Ak illAl ot io.u ci3lAc rc tui A3ta3tquotesdbs_dbs17.pdfusesText_23
[PDF] Data Mining: Overview What is Data Mining?