[PDF] Calculation ofsignal detectiontheory measures

A brief introduction of Signal Detection Theory

Sensitivity and Criterion •Performance is a function of TWO properties of the observer: •How well the observer perceives stimuli –or sensitivity •And, how does the observer choose to respond – or criterion •We need methods to assess the two independently •SIGNAL DETECTION THEORY 11

Statistical Decision Theory Signal Detection Theory

Signal Detection Theory: Criterion SDT: Gaussian case N S+N x ty 0 c d′ c=z[p(CR)] d′=z[p(H)]+z[p(CR)]=z[p(H)]−z[p(FA)] z[p(CR)] z[p(H)] G(x;µ,σ)= 1 2πσ e−(x−µ)2/2σ2 β= p(x=cS+N) p(x=cN) = e−(c−d′)2/2 e−c2/2 (Fix = 1) Internal response ty False Alarms Hits 0 1 1 0 ROC (Receiver Operating Characteristic) Criterion #1

Signal Detection Theory - Center for Neural Science

Signal Detection Theory Professor David Heeger November 12, 1997 The starting point for signal detection theory is that nearly all decision making takes place in the presence of some uncertainty Signal detection theory provides a precise lan-guage and graphic notation for analyzing decision making in the presence of uncertainty Simple Forced

Signal Detection Theory - University of British Columbia

Signal Detection Theory 1 The problem: Theory: Data: There doesn’t seem to be a clear absolute (or differential) threshold Correction for guessing doesn’t help There are cases where there is no stimulus present but the subject perceives something => noise 2 Proposal: Signal detection is a signal /noise decision problem 3

Sensitivity and Bias - an introduction to Signal Detection

Signal Detection Theory (SDT) offers a framework and method for doing this, and in general for distinguishing between the sensitivity or discriminability (d') of the observer and their response bias or decision criterion (C) in the task

SIGNAL DETECTION THEORY: CONSIDERATIONS FOR GENERAL APPLICATION

Jan 16, 1974 · signal detection theory can be applied, the range of applications possible, or the limitations of signal detection theory This paper briefly summarizes the assumptions of signal detection theory and describes the procedures, the limi-tations, and practical considerations relevant to its application A worked

Calculation ofsignal detectiontheory measures

Signal detection theory (SOT) is widely accepted by psychologists; the Social Sciences Citation Index cites over 2,000 references to an influential book by Green and Swets (1966) that describes SOT and its application to psychology Even so, fewerthan halfofthe studies to which SOT isapplicable actuallymakeuseofthetheory (Stanislaw & Todorov

Lecture 12 Signal detection theory - UVicca

Today’s handout illustrates several concepts of the signal detection theory in the context of airline security procedures After reading it, your team should find answers to the following questions (short answers to 1-4, longer answer to 5) • What elements were perceived by security officials that indicated “SIGNAL PRESENT”? •

[PDF] signal detection theory d'

[PDF] signal electrique definition

[PDF] signal electrique exemple

[PDF] signal electrique periodique

[PDF] Signal périodique enregistré ? l'oscilloscope

[PDF] signaler un problème facebook

[PDF] signature d'une forme quadratique exemple

[PDF] Signature du traité de Versailles de William Orpen

[PDF] signature porcelaine ancienne

[PDF] signatures faiences francaises

[PDF] signaux electrique cours

[PDF] signaux periodique definition

[PDF] signaux periodiques cardiaques 2nd

[PDF] signaux sonores cycle 4

[PDF] signe accouchement approche

Behavior Research Methods, Instruments,&Computers

1999,3/(I),/37-149

HAROLDSTANISLAW

and

NATASHATODOROV

Signaldetectiontheory (SDT)may be applied to anyarea ofpsychology in which two different types ofstimuli must be discriminated. Wedescribe several ofthese areas and the advantagesthatcan be re alizedthroughthe application of SDT.Three of the mostpopulartasks used to study discriminability are then discussed,togetherwith themeasuresthatSDTprescribesfor quantifyingperformancein these tasks. Mathematicalformulae for themeasuresarepresented,as aremethodsfor calculating the measureswith lookup tables,computersoftware specificallydeveloped for SDTapplications,and gen eralpurposecomputersoftware(includingspreadsheetsandstatisticalanalysis software).

Signaldetectiontheory (SOT) iswidelyacceptedby

psychologists;the

SocialSciencesCitationIndexcites

over 2,000referencesto an influential book by Green and

Swets (1966)thatdescribes

SOT and itsapplicationto

psychology.Even so, fewerthan halfofthe studies towhich

SOTisapplicableactuallymakeuse

ofthetheory (Stanislaw &Todorov, 1992). One possible reason for this apparent underutilization ofSOT is that relevanttextbooksrarely describe themethodsneeded toimplementthe theory. A typical example isGoldstein's(1996)popularperception textbook, which concludes a nine-page description ofSOT with thestatementthat measuresprescribedby SOT "can becalculated ...by meansofamathematicalprocedure we will not discuss here" (p. 594).

The failure

ofmany authors to describe SOT's methods may havebeen acceptable when lengthy,specializedtables wererequiredtoimplementthe theory. Today, however, readily availablecomputersoftware makes an SOT analy sis no moredifficultthan a ttest. Thepresentpaper at tempts todemonstratethis and to render SOT available to a larger audience than currently seems to be the case.

We begin with a

briefoverviewofSOT,includinga description ofitsperformancemeasures. We then present the formulae needed to calculate these measures. Next,we describe different methods for calculating SOT measures. Finally, weprovidesamplecalculationsso thatreaders can verify theirunderstandingandimplementationofthe techniques. Weare indebted to James Thomas, Neil Macmillan, John Swets, Doug lasCreelman,ScottMaxwell,MarkFrank,HelenaKadlec,and an anonymousreviewer for providing insightfulcommentson earlier ver sions ofthismanuscript.We alsothankMackGoldsmithfortesting some article should beaddressedto H. Stanislaw,Department ofPsychology, CaliforniaState University, Stanislaus, 801 West Monte Vista Avenue, Turlock, CA 95382 (e-mail:hstanisl@toto.csustan.edu).

OVERVIEWOFSIGNAL

DETECTIONTHEORY

Proper applicationofSOT requires an understanding of thetheoryand themeasuresitprescribes.We present an overview ofSOT here; for more extensive discussions, see Green andSwets(1966)orMacmillanandCreelman (1991). Readers who are already familiar with SDT may wish to skip this section.

SOT can beapplied whenevertwopossiblestimulus

types must be discriminated. Psychologists first applied the theory in studies ofperception,where subjects discrimi nated between signals(stimuli) andnoise(no stimuli). The signal and noise labels remain, but SOT has since been ap pliedin manyotherareas.Examples(andtheircorre spondingsignal and noisestimuli)includerecognition memory (old and new items), lie detection (lies and truths),

personnel selection (desirable and undesirable applicants),jurydecisionmaking(guiltyandinnocentdefendants),

medical diagnosis (diseased and well patients), industrial inspection(unacceptableandacceptableitems), and in formationretrieval (relevant andirrelevantinformation; see also Hutchinson, 1981; Swets, 1973;and the extensive bibliographiescompiled by Swets, 1988b, pp.685-742).

Performancein each

ofthese areas may be studied with a variety oftasks. Wedeal here with threeofthe most popu lar:yes/notasks,ratingtasks, andforced-choicetasks.

YeslNo Tasks

A yes/no task involves

signal trials,which present one or moresignals,and noise trials,whichpresentone or more noise stimuli. For example, yes/no tasks inauditory perceptionmaypresenta toneduringsignaltrialsand nothing at all during noise trials, whereas yes/no tasks for memory may present old(previously studied) words dur ing signal trials and new(distractor)words during noise trials. After each trial, the subjects indicate whether a sig-

137Copyright 1999PsychonomicSociety, Inc.

138 STANISLAWANDTODOROV

d' "0 oo Q) en ·0 Z 432o

Criterion

-1-2 ................,;:i:IIIpo...........,., -3.4 Noise distribution .3 :.0 m .2 .0 0 0... .1

Decision variable

d';c, andthelikelihoodsonwhichf3isbased. nal waspresented(i.e.,whethera tone waspresented,or whetherthe word waspreviouslystudied).

Accordingto SDT, thesubjectsin ayes/notaskbase

their response on the value that a decision variableachieves during each trial.

Ifthedecisionvariableissufficiently

high during a given trial, the subject responds yes(a signal no(no signal waspresented).The value thatdefines sufficiently high iscalledthecriterion.

Forexample,considerapsychologistattemptingto de

terminewhetheror not achildhasattentiondeficithyper activitydisorder(ADHD).Thepsychologist mightad minister the ChildBehaviorChecklist (CBCL;Achenbach &Edelbrock,1983) anddiagnosethechildashaving ADHD if theresultingscore is 10or higher (Rey,Morris

Yates,

&Stanislaw, 1992). In this case, thedecisionvari able is the CBCL score, and thecriterionis set at 10. In the ADHDexample,the decision variable is readily observed. However,most ofthe tasks studied bypsycholo gists involvedecisionvariablesthat areavailableonly to the subjectperformingthe task. For example, the decision variable may be theapparentloudnessexperiencedduring each trial in anauditoryperceptionstudy, thefeeling of familiarity associated with each stimulus item in amemory study, or theapparentguilt ofeachdefendantin astudyof jurydecision making. In eachofthese cases, thesubjects comparethedecisionvariable(which only they can ob serve) to thecriterionthey have adopted. A yesresponse is made only if theauditorystimulusseemssufficiently loud, the stimulus item seems sufficiently familiar, or the defendantseemssufficientlyguilty.

On signal trials,

yesresponses are correct and are termed hits.On noise trials,yesresponsesareincorrectand are termedfalsealarms.Thehit rate(theprobabilityofre spondingyeson signal trials) andthefalse-alarmrate(theprobability ofrespondingyeson noise trials) fully describe performanceon a yes/no task. Ifthesubjectis using anappropriatedecisionvariable, and ifthesubjectis capableofdistinguishingbetween sig nals and noise, the decision variable will be affected by the stimuli that arepresented.Forexample,previouslystudied words inamemorystudy should,onaverage, seem more fa miliarthandistractors.However, somepreviouslystudied words willseemmorefamiliarthanothers.Distractors will alsovaryintheirfamiliarity.Furthermore,factors such as neuralnoiseandfluctuationsinattentionmay af fect thedecisionvariable, even if thestimulusis held con stant. Thus, the decision variable will have a range ofdif ferent valuesacrosssignal trials and arange ofdifferent values across noise trials. (For moreexamples ofthis, see

McNicol,1972, pp.11-14.)

Thedistribution

ofvaluesrealizedby thedecisionvari able acrosssignaltrials is the signaldistribution,whereas thecorrespondingdistributionfor noise trials is the noise distribution.

The hit rate equals theproportionofthe signal

distributionthatexceedsthecriterion,whereasthe false alarm rateequalstheproportion ofthe noise distribution thatexceedsthecriterion.This isillustratedin Figure I, where thedecisionvariable(measuredinarbitraryunits) has a mean of0 and a standarddeviationof1on noise tri als. On signal trials, the mean ishigher (M=2), but the standard deviation isunchanged.

Ayesresponse is made for

trials in which thedecisionvariableexceeds0.5; these tri als lie in theshadedregion ofthe twodistributions.The shaded region ofthe noisedistributionconstitutes30.85% ofthe entire noise distribution, so thefalse-alarmrate is .3085. Bycontrast,93.32% ofthesignaldistributionis shaded, so the hit rate is .9332. Ifthecriterionis set to an even lower, or moreliberal, value (i.e., moved to the far left in Figure I), it will almost always be exceededonsignaltrials.Thiswillproduce mostly yesresponsesand a high hit rate. However, the cri terion will also beexceededon mostnoisetrials,resulting in a highproportion ofyesresponses on noise trials (i.e., a highfalse-alarmrate). Thus, a liberalcriterionbiases the subjecttowardresponding yes,regardlessofthe stimulus.

Bycontrast,ahigh,or

conservative,value for thecrite rion biases thesubjecttowardresponding no,becausethe criterionwillrarelybeexceededonsignalor noise trials. This will result in a lowfalse-alarmrate, but also a low hit rate. The only way toincreasethe hit ratewhilereducing thefalse-alarmrate is toreducetheoverlapbetweenthe signal and thenoisedistributions. Clearly, the hit andfalse-alarmratesreflecttwo factors: response bias(thegeneraltendencytorespondyesorno, asdeterminedby thelocationofthecriterion)and the de gree ofoverlapbetweenthe signal and thenoisedistribu tions. Thelatterfactorisusuallycalled sensitivity,re flecting theperceptualorigins ofSDT:Whenanauditory signal is presented, the decision variable will have a greater value (thestimuluswillsoundlouder)inlistenerswith more sensitivehearing.Themajorcontribution ofSDT to psychologyis theseparation ofresponsebias and sensi tivity. This point is so critical that weillustrateit with five examplesfromwidelydisparateareas ofstudy. higherthresholdshave beenreportedforswearwords than for neutralstimuli(Naylor &Lawshe, 1958). Asomewhat

Freudianinterpretation

ofthisfindingis that itreflects tion"againstnegativestimuli(Erdelyi,1974). However, a falsealarmismoreembarrassingfor swear words than for neutral stimuli.Furthermore,subjectsdo notexpectto en counterswear words in a study and are,therefore,cautious aboutreportingthem.Thus,different apparentthresh olds fornegativethan forneutralstimulimay stem from differentresponsebiases, as well as fromdifferentlevels ofsensitivity. Inordertodeterminewhichexplanationis separately. sometimesimprovesrecall, but it alsoincreasesthe num ber ory, orwhetherdemandcharacteristicscausehypnotized subjects toreportmorememoriesabout which they are un certain (Klatzky &Erdelyi, 1985). Theformerexplanation implies an increase in sensitivity, whereas the latter implies thathypnosisaffectsresponsebias. Again, it isimportant

Our thirdexampleinvolves aproblemthatsometimes

arises whencomparingtheefficacy oftwo tests used to diagnosethesamemental disorder. One test may have aquotesdbs_dbs1.pdfusesText_1