[PDF] Unsupervised Editing for Counterfactual Stories

View - Download Unsupervised Editing for Counterfactual Stories

Previous PDF

Next PDF

UnsupervisedEditing for CounterfactualStories

Jiangjie Chen

1,2*, ChunGan 3*, SijieCheng 1, HaoZhou 2†, YanghuaXiao1,5†, LeiLi 4‡

1 Shanghai KeyLaboratoryofData Science,School ofComputer Science,Fudan Univ ersity

2ByteDance AILab 3JD.com4UniversityofCalifornia, SantaBarbara

5Fudan-Aishu CognitiveIntelligenceJointResearch Center

f jjchen19, sjcheng20,sha wyh g @fudan.edu.cn, cgan5@wisc.edu,zhouhao.nlp@bytedance.com, lilei@cs.ucsb.edu

Abstract

Creatingwhat-ifstories requiresreasoning aboutprior state- ments andpossible outcomesof thechanged conditions.One can easilygenerate coherentendings underne wconditions, butit would bechallengingforcurrent systems todo itwith minimal changesto theoriginal story. Therefore,one major challenge isthe trade-off betweengeneratingalogical story and rewritingwithminimal-edits. Inthis paper, wepropose EDUCAT, anediting-based unsupervisedapproach forcoun- terfactualstory rewriting. EDUCATincludes atar getposition detection strategybasedon estimatingcausal effects ofthe what-ifconditions, whichk eepsthecausalin variant partsof the story.EDUCATthen generatesthe storiesunder fluency , coherence andminimal-edits constraints.W ealso proposea newm etrictoalleviate theshortcomings ofcurrentautomatic metrics andbetter ev aluatethetrade-off.We ev aluateEDU- CATon apublic counterfactual storyrewritingbenchmark. Experiments showthatE DUCATachievesthebest trade-off overunsupervisedSO TA methodsaccordingtobothauto- matic andhuma nevaluation. TheresourcesofEDUCATare availableat:https://github .com/jiangjiechen/EDUCAT.

1 Introduction

Counterfactual reasoningis ah ypotheticalthinkingprocess to assesspossible outcomesby modifyingcertain priorcon- ditions. Itis commonlykno wnas "what-if"analysis- "what willhappen if. .. ".Itisabigchallengetob uildan intelligent systemwith counterfactual reasoningcapabili- ties (Pearl2009; Pearland Mackenzie 2018).Counterf actual reasoning relieson theability tofind thecausal invariance in data,i.e. thef actorsheld constantwiththechange ofcon- ditions ina seriesof ev ents(Sloman andLagnado2004). In thispaper ,westudyunsupervisedcounterfactualstory rewriting,a concreteinstance ofcounterf actualreasoning. Wefocus onunsupervisedmethods forthis task,since hu- mans donot needsupervised learningto imaginealternati ve futures. Thetask isto createplausible alternativ eendings givensmallmodifications tothe storyconte xt.*

Workis doneduring internshipat ByteDanceAI Lab.

†Corresponding authors. ‡Workis donewhile atByteDance AILab . Copyright© 2022,Association forthe Advancement ofArtificial Intelligence (www.aaai.org).Allrightsreserv ed.SÕ2: Kelly never beat the game though S3: She was playing for so long without beating the level. S4:

Finally

she beat the last level.

S5: Kelly was so

happy to inally beat it. SÕ3: She was playing for so long without beating the level.

SÕ4: She

never beat the last level.

SÕ5: Kelly was so

sad to be stuck at the end

S1: Kelly was playing

her new Mario game.

Counterfactual

Storyline

S2: She

had been playing it for weeks

Original

Storyline

SÕ3: She was playing for so long

without beating the level.

SÕ4: She beat

never beat the last level.

SÕ5: Kelly was so happy to

inally beat it.

SÕ3: She was playing for so long

without beating the level.

SÕ4: She never beat the last level.

SÕ5: Kelly was so happy

sad to inally beat it.

What ifÉ

Iterative Editing

by g(x t +1 !x t x t x t +1

Original Ending

Counterfactual Ending

Step1:

Accept

Step2:

Accept

Step3:

Reject

Step4:

Reject

Step5:

Accept

É.Figure 1:Counterf actualstoryrewriting example fromthe TIMETRAVEL(Qin etal. 2019)dataset. Ourproposed EDU- CATiterativelyeditsthe originalending toobtain new end- ings. In thistask, themajor challengeis thetrade-of fbetwe en generatingnaturalstories andmodifying theoriginal text withminimal-edits. Thisrequires findingthe causalin vari- ance ina story, i.e.,invariant futuree ventsunderthechange of conditions.Indeed,with apre-trained languagemodel (LM), itis relativ elyeasytogeneratefluentendings un- der newconditionswith massive edits. However,difficul- ties arisewhen onehas toperform accuratereasoning during modifying theending minimallywhile keepingitnatural. Fore xample,inFigure1, whatif Kelly playedwith the

Mario gamebut neverbeat thegame (alters2

tos02)? From human commonsense,onecaneasily createaplausiblealter - nativestoryending bymaking smalledits thatK ellynever beat thelast lev elratherthannallybeat it,and henceK elly wouldbe sadinstead ofhappy. Inthis case,the invariant eventisthat Kelly stillplays alllevels untilthe last,b utthe variante ventwouldbethe consequenceofthecounterf ac-

tual intervention.Byidentifying andk eepingthe inv ariantTheTh irty-SixthAAAIConference onArtifi cialIntelligence(AAAI-22)

10473
event,anideal systemcan generatea plausibleending with fewedits tothe variant ev ents.

Most ofthe existing methods(Li,Ding,and Liu2018;

Xu etal. 2018;Guan, Wang, andHuang 2019;Guanetal.

2020) focuson thestory generationin anauto-re gressiv e

manner.These approachesk eepthe storylogicalmainlyby exploitingthe languagemodeli ngability ofLMssuchas the GPTs (Radfordet al.2018, 2019;Bro wnet al.2020). Few of them(Qin etal. 2019,2020) dealwith thereasoning abil- ity incounterf actualtextgeneration, whichrequiresbalanc- ing betweencoherence andminimal-edits. For example, Qin et al.(2020) proposeto keep thebalance byconstrainingthe decoding onne wendingswitha sentence-lev elsimilarity scorer withthe originalones. Howe ver ,LMsareknownto be hardto control,often leadingto ov er-editing.

In thispaper ,weproposeE DUCAT, anEDiting-based

U nsupervisedCounterfactualgener ATion methodfor coun- terfactualstory rewriting. Giventhe originalstoryanda modied conditionstatement, thechallenge isto locate which partto retain(i.e. causalin variance) andwhich to modify(i.e. causalv ariance)while maintainingcoher- ence tothe context afterediting.Inspiredby causalanaly- sis research(Hern

´an 2004),we quantifythe potentialout-

come afterinterv entionusingtheratio betweenconsisten- cies withthe counterfactual andinitialconditions,which can be computedby anof f-the-shelfmodel. EDUCATemploysa MarkovchainMonte Carlosampling framew ork(Metropo- lis etal. 1953)for unsupervisedgeneration byiterati vely generating tokenmodications(Miao etal. 2019).W ithde- sired propertiesand guidancefrom theestim atedpotential outcome, EDUCATgenerates uentand coherentalternati ve story endingswith minimaledits.

The contributionsofthis work areas follows:

• Werstsolv ethe counterfactualstory rewriting task using unsuperviseddiscrete editingmethod basedon

MCMC sampling.

• Wedraw inspirationfromcausal analysisandpropose twocounterf actualreasoningcomponentsthat quantify the outcomesof context changes. • Weconducte xperimentsto verifythatE DUCATachieves the besttrade-of fbetweencoherenceand minimal-edits for unsupervisedmethods.

2 TaskFormulation withCausalModel

In counterfactualstoryre writingtask, givena storyconsist- ing ofa premisez, astory context xand anending y, we interveneby alteringxinto acounterf actualcontextx0and hope topredict new endingy0. This problemnaturally tsto beformulated witha Causal Model , adirected acyclic graphusedtoencode assumptions on thedata generatingprocess. Aspresented inthe Figure2, the leftpart shows asimpleexample ofa causalm odelwith treatment(X),effect(Y) andconfounder(Z), respectively. In causalinference, aconfounder isa randomv ariablethat inuences boththe treatmentand effect variables, causinga spurious correlation(Pearl 2009).Note thatin thisproblem, zconsists ofboth observed confounders1and unobserved commonsense knowledge,wherethe latteris very difcult XY Z xy z x ! y! z

Prediction

Intervention

Confounder

TreatmentEffectFigure2:Formulatingcounterfactualstoryrewritingwithin-terventionon causalm ode l,wherezis thecommon premise

of thestory ,x;ydenote theoriginal story ,andx0;y0are the counterfactualstory . to explicitlymodel. The counterfactualinferencecan beformulated witha do- operator.As shown inFigure2,we caninterv eneon theX variableby applyingdo (X) =x0to setits value tothe counterfactualwithout changingthe rest.The arrow point- ing fromZtoXin thecausal modelis deletedsince X no longerdepends onZafter theinterv ention,resultingina newgraphical model.Consequently ,the problemofcounter- factualstory generationcan beformally restatedas acoun- wouldthe potentialoutcome ofybe ifone changesthe story contextfrom xtox0?

3 ProposedAppr oach:EDUCAT

In thissection, wepresent ano vervie wand detailsofED- UCAT. Ingeneral, there writingprocess worksasfollo ws: starting withan originalfull story, EDUCATperforms the followingprocedures iteratively:

1.Conict Detection, itnds possiblechunks incurrent story endingscontradictory tocounterf actualconditions;

2.Edits Proposal, itproposes anedited endingand decidesits acceptancebased onuenc yand coherencescores.

The abovestepsrepeatmultiple rounds.Each proposalis either acceptedor rejectedbased ondesired properties(y), which isdened asthe scoreproduct ofeach propertyscore: (y)/Desired Properties z}|{ X

0c(y)Xnc(y)(1)

Finally,we pick thebestoneaccording toa rankingfunction as theoutput. Anillustrati ve exampleisgiven inFigure1.

However,thechallengeremainsfor thequantication of

these desiredpropertiesfor idealstory rewriting. Inspired by causalanalysis research,we canquantitati vely calculate the differenceofstory endings'quality giv endif ferentcon- ditions withthe CausalRisk Ratio(CRR )(Hern

´an 2004;

Hern

´an andRobins 2020).CRR isdened asfollo ws:

CRR =

The valuegoesup whenthe new endingis moreconsis-

tent withthe counterfactual condition.Howev er, itisdif- cult toe xplicitlycalculatebothobserv edand unobserved confounders ( z ?) inP(Y=yjdo(X=x))as follows:

P(Y=yjdo(X=x))z}|{

X z ?P(Y=yjX=x;Z=z?)P(Z=z?)(3) Wemak eacausalsuf ciency assumptionthat onlyobserved confounder ( z) isconsidered:

P(Y=yjdo(X=x)) =P (Y=yjX=x;Z=z)(4)

quotesdbs_dbs25.pdfusesText_31

[PDF] Beat It Uptight Es-tu prête pour un bon martèlement bébé ? (a terre

[PDF] Beata de Robien

[PDF] beatclub-greven.de | BC Lounge

[PDF] beatclub-greven.de | Manfred Mann`s Earthband

[PDF] beatclub-greven.de | Radio

[PDF] Beate et Serge KLARSFELD

[PDF] Beati omnes - Alliance Music Publications

[PDF] Beati Quorum Via - Choral Public Domain Library

[PDF] Béatification du Pape Jean-Paul II : Corsica Ferries emmène 700 - Anciens Et Réunions

[PDF] beatification Jean

[PDF] beatificazione dei servi di dio: charles de foucauld

[PDF] béatifié - Page d`accueil la Mésange - La Religion Et La Spiritualité

[PDF] Beatiho :Fiche Technique Son

[PDF] Beating Cheating: Teachers and the Capital Intellectual Crime - Anciens Et Réunions

[PDF] Beatlemania - photo12.com