[PDF] Transfer Learning methods for temporal data

View - Download Transfer Learning methods for temporal data

Previous PDF

Next PDF

>G A/, i2H@yj88y9R8 ?iiTb,ffi?2b2bX?HXb+B2M+2fi2H@yj88y9R8 h`Mb72` G2`MBM; K2i?Q/b 7Q` i2KTQ`H /i hQ +Bi2 i?Bb p2`bBQM,

Thèse de doctorat

NNT: 2021UPASM037Methodes d'apprentissage statistique de type "Transfer Learning" pour des donnees temporelles multivariees

Transfer Learning for Temporal Data

These de doctorat de l'Universite Paris-Saclay

Ecole doctorale n574, Ecole Doctorale de Mathematiques Hadamard (EDMH)

Specialite de doctorat: Mathematiques Appliquees

Unite de recherche: Centre Borelli

Referent: ENS Paris-Saclay

These presentee et soutenue a Gif-sur-Yvette,

le 7 octobre 2021, par

Guillaume RICHARDComposition du jury

Younes BennaniPresident et RapporteurProfesseur, Universite Sorbonne Paris Nord (Laboratoire

d'Informatique de Paris NordGianluca BontempiRapporteurProfesseur, Universite Libre de Bruxelles (Departement

d'Informatique)Jairo CugliariExaminateurMaitre de Confeerences, Universite Lumiere 2 (Laboratoire ERIC)

Michele SebahExaminatriceDirectrice de Recherche, INRIA (Equipe TAO)

Direction de la these

Mathilde MougeotDirectriceProfesseur, ENS Paris-Saclay (Centre Borelli) et ENSIIE Nicolas VayatisDirecteurProfesseur, ENS Paris-Saclay (Centre Borelli) Georges HebrailTuteur en entrepriseChercheur Senior, EDF R&D

Remerciements

Mes premiers remerciements vont

`a mes directeurs de th`ese, Mathilde Mougeot et Nicolas Vayatis pour leur con- fiance au long de ces trois ann ´ees de th`ese, et de m"avoir donn´e l"opportunit´e de travailler sur des sujets aussi int

´eressants. Je remercie´egalement Georges H´ebrail qui a su m"accompagner d`es mes premiers pas chez EDF

et d

´epasser son rˆole d"encadrant industriel pour toujours me donner des bons conseils. Je remercie´egalement

tous les membres du jury pour avoir accept ´e de participer`a ma soutenance de th`ese. En particulier, je remercie Youn

`es Bennani qui a accept´e de pr´esider le jury en plus d"avoir rapport´e ma th`ese, Gianluca Bontempi qui a lui

aussi rapport ´e ma th`ese et enfin Jairo Cugliari et Mich`ele Sebag en tant qu"examinateurs.

Ensuite, je souhaite remercier toute l"

´equipe SOAD d"EDF pour tous les bons moments pass´es ensemble. J"aurai certainement l"occasion de le dire de vive voix, mais c" ´etait un r´eel plaisir de travailler avec vous. En plus de Georges, j"aimerais remercier Ghislain qui m"a accompagn ´e sur la fin de la th`ese. Quoiqu"il en soit, j"esp`ere qu"on sera amen ´e`a retravailler ensemble et que je serai toujours invit´e au Diamant! Je veux aussi remercier l"ensemble des membres du Centre Borelli, en particulier les th

´esards. L"entraide entre

les doctorants et nos discussions ont permis d" ´egayer les moments difficiles. En particulier, je remercie Antoine avec qui c" ´etait un plaisir de collaborer et je te souhaite le meilleur pour le reste de ta th`ese. Enfin, je veux remercier mes amis et ma famille, qui m"ont soutenu tout au long de la th `ese et m"ont aid´e`a me changer les id

´ees. En particulier, je remercie ma m`ere, mon p`ere et mon petit fr`ere qui m"ont fait grandir, m"ont d´ej`a

accompagn

´e dans toutes les´etapes importantes de ma vie et continueront`a le faire. Finally, I want to thank you,

Qu

`ynh Anh, for your love and support. It really made it easier to overcome every difficult moment and to move to

the next chapter together. i

Contents

R

´esum´e (en franc¸ais)11

1 Introduction19

1.1 Motivation

19

1.2 Organization of the manuscript

21

2 Background on Transfer Learning

25

2.1 What is Transfer Learning?

26

2.2 Theory of Domain Adaptation

28

2.2.1 Generalization bounds

29

2.2.2 Divergence-based Domain Adaptation

31

2.2.3 Alternative approaches

36

2.2.4 Summary

38

2.3 Existing approaches for Homogeneous Transfer Learning

38

2.3.1 Instance-based domain adaptation

39

2.3.2 Feature-based domain adaptation

42

2.3.3 Alternative methods

44

2.3.4 Summary

45

2.4 Learning from Multiple Sources

46

2.4.1 Multi-Task Learning and Domain Generalization

47

2.4.2 Multi-source domain adaptation

49

2.5 Conclusion

51

3 Deep Time Series Representations for Non-Intrusive Load Monitoring

53

3.1 Background on Time Series Representations

55

3.1.1 Framework

55

3.1.2 Overview of Univariate Time Series Representations

55
1

3.2 Transferability of Deep Time Series Representations. . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

3.2.1 Deep Time Series Representations

58

3.2.2 On Transferability of Deep Time Series Representations

61

3.3 Transfer Learning in Non Intrusive Load Monitoring

61

3.3.1 General presentation

61

3.3.2 Review of methods

63

3.3.3 Datasets

64

3.3.4 Problem formulation

66

3.4 Time Series Normalization for Invariant Appliance Recognition

68

3.4.1 Global and z-normalization

68

3.4.2 Normalization for appliance consumption

70

3.4.3 Model

72

3.5 Experiments on NILM Datasets

73

3.5.1 Preprocessing and Methods

74

3.5.2 Same House

77

3.5.3 Cross-House Results

77

3.5.4 Cross-Dataset Results

79

3.5.5 Discussion

81

3.6 Conclusion

82

4 Domain adaptation with multiple sources in regression

85

4.1 Domain Adversarial Learning withH-divergence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

4.1.1 Literature review

87

4.1.2 Limits of Domain Adversarial Adaptation in Regression withH-divergence. . . . . . . . . . . . 88

4.2 Hypothesis-Discrepancy for Domain Adaptation in Regression

89

4.2.1 Hypothesis-Discrepancy

89

4.2.2 Domain Adaptation Guarantees with Hypothesis-Discrepancy

91

4.3 Minimizing the hypothesis-discrepancy

92

4.4 Extension to multiple sources

94

4.4.1 Theoretical Guarantees with multiple sources

94

4.4.2 Algorithm

97

4.5 Experiments

98

4.5.1 Synthetic data

99

4.5.2 Appliance Consumption Estimation

102
2

4.5.3 Same-house results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

4.5.4 Cross-house results

105

4.5.5 Experiments on other datasets

108

4.6 Extension to semi-supervised adaptation

111

4.7 Conclusion

114

5 Covariance-based Transfer Learning with applications to Multivariate Time Series

115

5.1 Outline of the method

116

5.2 Multivariate Time Series and Covariance

118

5.3 Riemannian Geometry of Symmetric Positive Definite Matrices and Time Series

120

5.3.1 Basics

120

5.3.2 Working with time series

122

5.3.3 Statistical Learning with SPD Matrices

123

5.4 Transferable subspace using Covariance information

123

5.4.1 Framework

123

5.4.2 Learning a subspace aligning domains

124

5.4.3 Related works

127

5.4.4 Algorithm

128

5.4.5 Hyperparameter Selection

130

5.5 Numerical Results

132

5.5.1 Simulated data

132

5.5.2 Human Activity Recognition

135

5.6 Conclusion

141

6 Conclusion and Perspectives143

A Neural Networks147

B Clustering consumer consumption with auto-encoders 151

B.1 Data presentation

151

B.2 Method

152

B.2.1 Convolutional AutoEncoder

152

B.2.2 Compared methods

153

B.2.3 Outliers

154

B.3 Results

155
3

C Implementations159

C.1 Public implementations

159

C.2 Other implementations

160
C.2.1 List of statistical features extracted (Chapter 3 160

C.3 Additional Experiments of Chapter

3 162

C.4 Details about implementations of Chapter

4 163
4

List of Figures

1 Illustration du NILM:

`a partir de la consommation totale de la maison, l"objectif est de retrouver la consommation de chaque appareil. 13

1.1 Non Intrusive Load Monitoring illustration: from the whole house consumption, the goal is to retrieve

the consumption of each appliance 20

2.1 Taxonomy of Transfer Learning

27

2.2 Illustration of theH-divergence with linear classifiers. . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

2.3 Illustration of the discrepancy with linear regressorsH=fh:x!wTx;kwk21gand`2-loss.

Source and target data are generated as 1D-Gaussian distributions centered on1and+1with

different standard deviations in each graph (f1;1g,f0:5;5gandf4;4g).. . . . . . . . . . . . . . . . . . 36

2.4 Domain Adversarial Neural Network (figure from [

72
44

2.5 Scenarios of Transfer Learning with or without multiple sources

51

3.1 Euclidean Distance vs Dynamic Time Warping

56

3.2 Different architectures for time series classification

59

3.3 Convolutional Auto-Encoder

60

3.4 Sub-problems of Non Intrusive Load Monitoring (NILM)

63

3.5 Examples of monitored appliances for each house: a blue square means the appliances was moni-

tored. A white square means it was not monitored or was mixed with another appliance. 65

3.6 Examples of monitored consumption for one day in the Electric DataBase (x-axis is in hours and

y-axis is in W) 66

3.7 Examples of monitored consumption for one day in the REFIT Database (x-axis is in hours and y-axis

is in W ; the house numbers correspond to the ones in the database as technical issues happened for other houses) 67

3.8 Examples of signatures from the Trace Base dataset (x-axis is in minutes, y-axis is in W). Signatures

have been zero-padded to a length of 2 hours. 69
quotesdbs_dbs27.pdfusesText_33

[PDF] Biographies des membres du jury - École Secondaire

[PDF] Biographies des membres du jury fédéral de la danse Présidente

[PDF] Biographies du Parc et du Château de Champs-sur

[PDF] Biographies EUROPE.rtf

[PDF] Biographies Felipe Sauvageon: Rumbas - i - Mexique Et Amérique Centrale

[PDF] Biographies individuelles

[PDF] biographies of our new partners/ associate partners/ directors.

[PDF] Biographies of speakers and moderators NAME - MGIEP - France

[PDF] biographies of the candidates

[PDF] Biographies: Policy Dialogue with Civil Society

[PDF] BIOGRAPHIE_files/CV site

[PDF] Biographie_files/Liste de mes expos - Anciens Et Réunions

[PDF] Biographie– Prof. Dr. Emmanuelle Charpentier - Max-Planck

[PDF] Biography -

[PDF] biography - Alison Jacques Gallery