[PDF] Nonlinear modeling of the guitar signal chain enabling its real-time

View - Download Nonlinear modeling of the guitar signal chain enabling its real-time

Previous PDF

Next PDF

Nonlinear modeling of the guitar signal

chain enabling its real-time emulation Effects, amplifiers and loudspeakersSchmitz Thomas

Supervisor: Prof. J.-J. Embrechts

Faculté des Sciences Appliquées

Département d"Électricité, Électronique et Informatique

Institut Montefiore

University of LiègeThis dissertation is submitted in fulfillment of the requirements of the degree of

Doctor of Philosophy in Engineering Sciences

University of Liège

October 2019

ii

© Schmitz Thomas 2019, All Right Reserved.

First printing, July 2019

DeclarationThe present dissertation has been evaluated by the members of the Jury (sorted by alphabetical

order): Dr .Stéphane DUPONT ,Uni versityof Mons, Belgium Prof. Jean-Jacques EMBRECHTS (advisor), Uni versityof Liège, Belgium Prof. Simone ORCIO NI,Marche Polytechnic Uni versity,Italy Prof. Pierre SA CRE,Un iversityof Liège, Belgium Prof. Maarten SCH OUKENS,Eindho venUni versityof T echnology,Netherlands Prof. Marc V ANDR OOGENBROECK(chair), Uni versityof Liège, Belgium I would like to dedicate this thesis to all musicians suffering while transporting their materials from one place to another.

AcknowledgementsWorking on this thesis during six years was a real pleasure. I learned a lot and I had the

opportunity to further a subject I care about. I honestly cannot imagine a better job. It was a fantastic experience and I would like to thank you all for making it possible. Of course, I cannot thank my parent enough. Raising a child is not an easy task but you did it well after only one attempt (just kidding, sis I love you). Anyway thank you for your help, patience and support. For this thesis, my greatest luck has been to have an awesome supervisor. Aside from your brilliant mind, I always really liked our open discussions and your availability. I have learned a lot from you and I will forever be grateful. Reading and evaluating a thesis is a task which demands times and energy, I would like to thank you, dear jury members, to have accepted it. I hope this could be the first step of a fruitful collaboration. I would like to thank the proof readers of my thesis: Jean-Jacques, Pierre and Aubéri. I also received a lot of support from my friends, which are always present to enjoy some time together as well in times of need. I also would like to thank the colleagues and the gardener of the Montéfiore institute who have made of this place a so pleasant place to work at. I am also grateful to the NVIDIA Corporation for the donation of a Titan Xp GPU to support this research. Last but not least, a special thanks Aubéri for your support, patience, love and delicious baking.

I wish you all a good reading,

Thomas

AbstractNonlinear systems identification and modeling is a central topic in many engineering areas since most real world devices may exhibit a nonlinear behavior. This thesis is devoted to the emulation of the nonlinear devices present in a guitar signal chain. The emulation aims to replace the hardware elements of the guitar signal chain in order to reduce its cost, its size, its weight and to increase its versatility. The challenge consists in enabling an accurate nonlinear emulation of the guitar signal chain while keeping the execution time of the model under the real-time constraint. To do so, we have developed two methods. The first method developed in this thesis is based on a subclass of the Volterra series where only static nonlinearities are considered: the polynomial parallel cascade of Hammerstein models. The resulting method is called the Hammerstein Kernels Identification by Sine Sweep method (HKISS). According to the tests carried out in this thesis and to the results obtained, the method enables an accurate emulation of nonlinear audio devices unless if the system to model is too far from an ideal Hammerstein one. The second method, based on neural networks, better generalizes to guitar signals and is well adapted to the emulation of guitar signal chain (e.g., tube and transistor amplifiers). We developed and compared eight models using different performance indexes including listening tests. The accuracy obtained depends on the tested audio device and on the selected model but we have shown that the probability for a listener to be able to hear a difference between the target and the prediction could be less than 1%. This method could still be improved by training the neural networks with an objective function that better corresponds to the objective of this audio application, i.e., minimizing the audible difference between the target and the prediction. Finally, it is shown that these two methods enable an accurate emulation of a guitar signal chain while keeping a fast execution time which is required for real-time audio applications.

RésuméL"identification et la modélisation des systèmes non linéaires sont des sujets majeurs dans

beaucoup de domaines de l"ingénieur. Cette thèse est dévouée à l"émulation des systèmes non

linéaires présents dans la chaîne de traitement du signal de la guitare. L"émulation a pour but

de remplacer les éléments matériels de la chaîne par leur équivalent numérique dans le but de

réduire son coût, sa taille, son poids et d"améliorer sa polyvalence. Le défi consiste à émuler

les éléments de la chaîne en temps réel en tenant compte de leur caractère non linéaire. Pour ce

faire, nous avons dévelopé deux méthodes.

La première méthode proposée est basée sur un sous-modèle de la série de Volterra où

seules les non-linéarités statiques sont considérées : la cascade de modèles d"Hammerstein mis

en parallèle. La méthode résultante que nous avons appeléeHammerstein Kernels Identification

by Sine Sweep(HKISS), rend possible l"émulation de systèmes audio non linéaires. Cette

méthode atteint ses limites quand le système à modéliser est trop différent du modèle idéal

d"Hammerstein. La variabilité des noyaux vis-à-vis de l"amplitude du signal d"entrée empêche

alors une émulation précise du signal. La seconde méthode proposée est basée sur l"utilisation de réseaux de neurones. La

méthode est plus appropriée aux signaux de guitare et s"adapte bien aux différents systèmes

nonlinéaires testés (circuits de distorsion, amplificateurs à tubes et à transistors). Nous avons

développé et comparé huit modèles de réseaux de neurones en utilisant différents indices de

performances incluant des tests d"écoutes. La précision obtenue dépend du modèle choisi et de

l"élément émulé mais nous avons montré que la probabilité qu"une personne puisse discerner

une différence entre le son de l"appareil testé et son émulation pouvait être inférieure à 1%.

Selon notre opinion, cette méthode pourrait encore être améliorée en entrainant les réseaux

de neurones avec une fonction-objectif qui correspond mieux à l"objectif de cette application

audio, à savoir, minimiser la différence audible entre le son de l"appareil testé et son émulation.

Ces deux méthodes permettent l"émulation fidèle d"une chaine d"instrumentation pour guitare, tout en gardant un temps d"exécution suffisamment bas pour respecter une contrainte temps réel acceptable pour ce genre d"application audio.

Contents

List of Figures

xix

List of Tables

xxvii

Nomenclature

xxix

I Introduction and Theoretical Background

1

1 Introduction

5

1.1 Emulation of a guitar chain . . . . . . . . . . . . . . . . . . . . . . . . . . .

5

1.1.1 Main concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5

1.1.2 Related works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

8

1.2 Contributions, objectives and applications . . . . . . . . . . . . . . . . . . .

11

1.3 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

13

1.4 Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

13

1.5 Download materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

14

2 Nonlinear models

15

2.1 Introduction to systems theory . . . . . . . . . . . . . . . . . . . . . . . . .

16

2.1.1 Systems and signals . . . . . . . . . . . . . . . . . . . . . . . . . .

16

2.1.2 Linear systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

17

2.1.3 Nonlinear systems . . . . . . . . . . . . . . . . . . . . . . . . . . .

18

2.2 General form of a nonlinear system . . . . . . . . . . . . . . . . . . . . . . .

19

2.3 Volterra model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

20

2.4 Block oriented models . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

25

2.4.1 Hammerstein model . . . . . . . . . . . . . . . . . . . . . . . . . .

25

2.4.2 Wiener model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

27
xivContents2.4.3 Sandwich or Wiener-Hammerstein model . . . . . . . . . . . . . . .28

2.4.4 Parallel cascade model . . . . . . . . . . . . . . . . . . . . . . . . .

30

2.4.5 Wiener-Bose model . . . . . . . . . . . . . . . . . . . . . . . . . . .

31

2.5 Nonlinear state-space model . . . . . . . . . . . . . . . . . . . . . . . . . .

32

2.6 NARMAX model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

33

2.7 Neural networks models . . . . . . . . . . . . . . . . . . . . . . . . . . . .

34

2.7.1 Activation function . . . . . . . . . . . . . . . . . . . . . . . . . . .

35

2.7.2 Feedforward neural networks . . . . . . . . . . . . . . . . . . . . .

38

2.7.3 Multi-layer neural networks . . . . . . . . . . . . . . . . . . . . . .

39

2.7.4 Recurrent neural networks . . . . . . . . . . . . . . . . . . . . . . .

40

2.8 Introduction to machine learning . . . . . . . . . . . . . . . . . . . . . . . .

41

2.8.1 What is machine learning ? . . . . . . . . . . . . . . . . . . . . . . .

42

2.8.2 Terminology and notation used in machine learning . . . . . . . . . .

46

II Emulation of the Guitar Signal Chain

49

3 Hammerstein kernels identification by exponential sine sweeps

55

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

56

3.1.1 Related works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

56

3.2 Hammerstein kernels identification method . . . . . . . . . . . . . . . . . .

57

3.2.1 Volterra series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

58

3.2.2 Cascade of Hammerstein models . . . . . . . . . . . . . . . . . . . .

58

3.2.3 Hammerstein kernels identification using theESSmethod . . . . . .59

3.2.4 Note on the Hammerstein model and its input level dependency . . .

66

3.3 Optimization of the Hammerstein kernels identification method . . . . . . . .

67

3.3.1Test algorithm to highlight the potential difficulties of the HKISS method67

3.3.2 The best case: a correct reconstruction through the HKISS method . .

69

3.3.3 Hammerstein kernels phase errors . . . . . . . . . . . . . . . . . . .

69

3.3.4 Phase mismatch when extracting the impulsesgm[n]. . . . . . . . .72

3.3.5ESSfade in/out . . . . . . . . . . . . . . . . . . . . . . . . . . . . .73

3.3.6 Spreading of the Dirac impulse . . . . . . . . . . . . . . . . . . . . .

73

3.3.7 Computing the powers of the input signal . . . . . . . . . . . . . . .

75

3.4 Emulation of nonlinear audio devices using HKISS method . . . . . . . . . .

77

3.4.1 Evaluation algorithm . . . . . . . . . . . . . . . . . . . . . . . . . .

77
Contentsxv3.4.2Emulation of a guitar distortion effect (tube screamer) through the HKISS method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

3.4.3 Emulation of the tube amplifier??????through the HKISS method .80

3.4.4 Emulation of the tube amplifier:Engl retro tube 50through the HKISS method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

3.5 Discussion on the limitations of the method . . . . . . . . . . . . . . . . . .

85

3.5.1 Order limitation . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

89

3.5.2 Low-frequency limitation . . . . . . . . . . . . . . . . . . . . . . .

89

3.5.3 Hammerstein kernels input level dependency . . . . . . . . . . . . .

91

3.6 Implementation with the Matlab toolbox . . . . . . . . . . . . . . . . . . . .

92

3.7 Example: emulation of a power series through the HKISS method . . . . . .

93

3.7.1 Sine sweep generation . . . . . . . . . . . . . . . . . . . . . . . . .

93

3.7.2 Hammerstein kernels calculation . . . . . . . . . . . . . . . . . . . .

94

3.7.3 Emulation of the DUT . . . . . . . . . . . . . . . . . . . . . . . . .

95

3.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

98

4 Emulation of guitar amplifier by neural networks

99

4.1 Introduction to the emulation of guitar amplifiers by neural networks . . . . .

100

4.1.1 Motivation to use neural networks for the guitar emulation task . . . .

100

4.1.2 Related works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

100

4.1.3 Dataset for guitar amplifiers . . . . . . . . . . . . . . . . . . . . . .

101

4.2 Different neural network models for guitar amplifier emulation . . . . . . . .

104

4.2.1 Model 1: Guitar amplifier emulations with a LSTM neural network .

105

4.2.2 Model 2: two layers of LSTM cells . . . . . . . . . . . . . . . . . .

121

4.2.3 Model 3: sequence-to-sequence prediction with one LSTM layer . . .

126
4.2.4 Model 4: taking the parameters of the amplifier into account in LSTM cells . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

4.2.5 Model 5: a faster LSTM model using convolutional input . . . . . . .

136

4.2.6 Model 6: feedforward neural network . . . . . . . . . . . . . . . . .

144

4.2.7 Model 7: convolutional neural network . . . . . . . . . . . . . . . .

148

4.2.8 Model 8: simple recurrent neural network . . . . . . . . . . . . . . .

153

4.3 Evaluation of the different neural network models through performance metrics

157
4.3.1 Comparison in the time domain of normalized root mean square error, computational time and signal to noise ratio . . . . . . . . . . . . . . 157
xviContents4.3.2Evaluation in the frequency domain based on spectrograms and power spectrums . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

4.4 Subjective evaluation of LSTM and DNN models by using listening tests . . .

164

4.4.1 The Progressive test: analysis of the PAD in function of the PI . . . .

164

4.4.2 The Hardest test . . . . . . . . . . . . . . . . . . . . . . . . . . . .

169

4.4.3 Choosing an appropriate objective function during the training phase

172

5 Conclusion and prospects

177

5.1 Summary of work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

177

5.2 Future works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

179

5.3 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

180

III Appendix

183

Appendix A More information about:

187
A.1 Back propagation algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 187
A.2 Overview of the methods improving the learnability of neural networks . . . 188
A.3 Dataset information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
A.3.1 Experimental setup . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
A.3.2 Building a learning dataset . . . . . . . . . . . . . . . . . . . . . . . 191
A.4 Vanishing or exploding gradient problem . . . . . . . . . . . . . . . . . . . . 193
A.5 Structure of the neural network models in Tensorflow . . . . . . . . . . . . . 194
A.5.1 Notation and basic informations: . . . . . . . . . . . . . . . . . . . . 195
A.5.2 Code defining the structure of the graph for model 1 . . . . . . . . . 196
A.5.3 Code defining the structure of the graph for model 2 . . . . . . . . . 198
A.5.4 Code defining the structure of the graph for model 3 . . . . . . . . . 199
A.5.5 Code defining the structure of the graph for model 4 . . . . . . . . . 200
A.5.6 Code defining the structure of the graph for model 5 . . . . . . . . . 202
A.5.7 Code defining the structure of the graph for model 6 . . . . . . . . . 204
A.5.8 Code defining the structure of the graph for model 7 . . . . . . . . . 205
A.5.9 Code defining the structure of the graph for model 8 . . . . . . . . . 207
Appendix B Additional developments for the third chapter 209

B.1 Development of Eq. (

3.7 209

B.2 Development of Eq. (

3.9 210

B.3 Development of Eq. (

3.15 212
ContentsxviiB.4 Matrix of phase correction presented in Sec.3.3.3 . . . . . . . . . . . . . . . 212

Bibliography

215

List of Figures

1.1 Stack of a guitar amplifier and its cabinet. . . . . . . . . . . . . . . . . . . .

7

1.2 Guitar effects in a rack and pedalboard. . . . . . . . . . . . . . . . . . . . .

7

1.3 Guitar signal chain setup . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7

2.1 Hammerstein model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

26

2.2 Wiener model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

27
quotesdbs_dbs25.pdfusesText_31

[PDF] CH5 - Les transformations chimiques

[PDF] Ch5 LA LOI D`OHM U (V) - Arithmétique

[PDF] CH52 0483 5012 3456 7100 0 L`IBAN Tous les

[PDF] Ch6 : Théorème des milieux 1 Propriété de la droite des milieux 2

[PDF] ch6 phénomènes d`induction - Tir À L'Arc

[PDF] ch6-Variables Instrumentales

[PDF] Ch6. Exercice corrigé. p : 174 n°15. Visée d`une fenêtre en lançant

[PDF] Ch7 : Division de fractions 1 Inverse 2 Propriétés des inverses 3 - Commercial Et Industriel

[PDF] CH7 – CM2 – Structure des Molécules Structure des Molécules

[PDF] Ch7 – Echantillonnage

[PDF] Ch7. Travail et énergie. Exercice résolu. p : 199 n°12. Pendule simple - Amélioration De L'Habitat Et De Réparation

[PDF] ch9 – la remise en banque des effets de commerce

[PDF] CH:OS:EN Saisons 1 à 3

[PDF] CHA -Youngtimers Cup - Qualifs - Achats

[PDF] Cha Cha Dancing - Festival