[PDF] SBaskar - IIT Bombay Numerical analysis is a branch

View - Download SBaskar - IIT Bombay

Previous PDF

Next PDF

Introduction to Numerical AnalysisS. Baskar

2

General Instructions

Course Number :SI 507

Course Title :Numerical Analysis

Course Syllabus

1.Mathematical Preliminaries:Continuity of a Function and Intermediate Value Theorem; Mean Value

Theorem for Differentiation and Integration; Taylor"s Theorem (1 and 2 dimensions).

2.Error Analysis:Floating-Point Approximation of a Number; Loss of Significance and Error Propagation;

Stability in Numerical Computation.

3.Linear Systems:Gaussian Elimination; Pivoting Strategy; LU factorization; Residual Corrector Method;

Solution by Iteration; Conjugate Gradient Method; Ill-Conditioned Matrices, Matrix Norms; Eigenvalue prob-

lem - Power Method; Gershgorin"s Theorem.

4.Nonlinear Equations:Bisection Method; Fixed-Point Iteration Method; Secant Method; Newton Method;

Rate of Convergences; Solution of a System of Nonlinear Equations; Unconstrained Optimization.

5.Interpolation by Polynomials:Lagrange Interpolation; Newton Interpolation and DividedDifferences;

Hermite Interpolation; Error of the Interpolating Polynomials; Piecewise Linear and Cubic Spline Interpola-

tion; Trigonometric Interpolation; Data Fitting and Least-Squares Approximation Problem.

6.Differentiation and Integration:Difference formulae; Some Basic Rules of Integration; Adaptive Quadra-

tures; Gaussian Rules; Composite Rules; Error Formulae.

7.Differential Equations:Euler Method; Runge-Kutta Methods; Multi-Step Formulae; Predictor-Corrector

Methods; Stability and Convergence; Two Point Boundary Value Problems.

Texts/References

1. K. E. Atkinson,An Introduction to Numerical Analysis(2nd edition), Wiley-India, 1989.

2. S. D. Conte and Carl de Boor,Elementary Numerical Analysis - An Algorithmic Approach(3rd edition),

McGraw-Hill, 1981.

General Rules

1. Attendance in lectures as well as tutorials is compulsory. Students not fulfilling the 80% attendance require-

ment may be awarded the XX grade.

2. Attendance will be recorded through an attendance sheet that will be circulated in the class. Each student

is expected to sign against his/her name only. Students who are found indulging in proxy attendance or any

form of cheating will be severely punished.

Evaluation Plan

1. There will be two quizzes (dates will be announced later),each of weightage 10% and one hour duration.

2. The Mid-Semester Examination scheduled during 11-18 September 2010 will be of 30% weightage.

3. The End-Semester Examination scheduled during 16-28 November will be of 40% weightage.

4. Lab assignments will be given through out the semester andthe students are expected to complete the

assignment and produce all the outputs asked at the end of thesemester. A oral viva will be conducted to

each student. The weightage will be of 10%.

5. To pass the course (DD), one needs to score minimum of 40% ofthe maximum mark scored in the class. For

instance, if the maximum mark scored is 80% at the end of the semester, then the passing mark will be 32%.

Higher grades will be based on the over all performance of theclass. Web Page: Course related materials will be uploaded in http://www.math.iitb.ac.in/baskar/baskar t.htm 3

Preface

In addition to the references provided above, class notes will be distributed in the class as a typed

material. These notes are meant only for SI 507 in Autumn 2010 as a supplementary material and cannot

be considered as a text book. Students are requested to refer the text books listed under course syllabus

for more details. These notes may have errors of all kind and the author request the readers to take care

of such error while going through the material. The author will be grateful to those who brings to his

notice any kind of error.

ContentsIntroduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . 7

1 Mathematical Preliminaries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 9

1.1 Continuity of a Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . 9

1.2 Differentiation of a Function . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.3 Integration of a Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.4 Taylor"s Formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Exercise 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2 Error Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . 17

2.1 Floating-Point Form of Numbers . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . 17

2.2 Chopping and Rounding a Number . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . 19

2.3 Different Type of Errors .. . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.4 Loss of Significant Digits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.5 Propagation of Error.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Exercise 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3 Linear Systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . 27

3.1 Gaussian Elimination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . 27

3.2 LU Factorization Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.3 Error in Solving Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . 32

3.4 Matrix Norm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.5 Iterative Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

3.6 Eigenvalue Problem: The Power Method. . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . 39

3.7 Gerschgorin"s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Exercise 3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

4 Nonlinear Equations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . 51

4.1 Fixed-Point Iteration Method . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 51

4.2 Bisection Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

4.3 Secant Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

4.4 Newton-Raphson Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

4.5 System of Nonlinear Equations . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . 60

4.6 Unconstrained Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . 62

6 Contents

Exercise 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

5 Interpolation by Polynomials. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 67

5.1 Lagrange Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

5.2 Newton Interpolation and Divide Differences . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . 69

5.3 Error in Polynomial Interpolation . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . 71

5.4 Piecewise Linear and Cubic Spline Interpolation . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . 75

Exercise 5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

6 Numerical Differentiation and Integration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

6.1 Numerical Differentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 79

6.2 Numerical Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

Exercise 6. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

7 Numerical Ordinary Differential Equations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

7.1 Review on Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 93

7.2 Discretization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

7.3 Euler"s Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

7.4 Runge-Kutta Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

7.5 An Implicit Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . 100

7.6 Multistep Methods: Predictor and Corrector . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . 101

Exercise 7. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . 105

Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

IntroductionNumerical analysis is a branch of Mathematics that deals with devisingefficient methods for obtaining

numerical solutions to difficult Mathematical problems. Most of the Mathematical problems that arise in science and engineering are very hard and sometime

impossible to solve exactly. Thus, an approximation to a difficult Mathematical problem is very impor-

tant to make it more easy to solve. Due to the immense development inthe computational technology, numerical approximation has become more popular and a modern tool for scientists and engineers. As a

result many scientific softwares are developed (for instance, Matlab, Mathematica, Maple etc.) to handle

more difficult problems in an efficient and easy way. These softwares contain functions that uses standard

numerical methods, where a user can pass the required parameters and get the results just by a single

command without knowing the details of the numerical method. Thus, one may ask why we need to understand numerical methods when such softwares are at our hands?

In fact, there is no need of a deeper knowledge of numerical methods and their analysis in most of the

cases in order to use some standard softwares as an end user. However, there are at least three reasons

to gain a basic understanding of the theoretical background of numerical methods.

1. Learning different numerical methods and their analysis will make aperson more familiar with the

technique of developing new numerical methods. This is important when the available methods are not enough or not efficient for a specific problem to be solved.

2. In many circumstances, one has more methods for a given problem. Hence, choosing an appropriate

method is important for producing an accurate result in lesser time.

3. With a sound background, one can use methods properly (especially when a method has its own

limitations and/or disadvantages in some specific cases) and, most importantly, one can understand what is going wrong when results are not as expected. Numerical analysis include three parts. The first part of the subject is about the development of a method to a problem. The second part deals with the analysis of the method, which includes the error

analysis and the efficiency analysis. Error analysis gives us the understanding of how accurate the result

will be if we use the method and the efficiency analysis tells us how fast we can compute the result.

The third part of the subject is the development of an efficient algorithm to implement the method as

a computer code. A complete knowledge of the subject includes familiarity in all these three parts. This

course is designed to meet this goal.

A first course in Calculus is indispensable for numerical analysis. The first chapter of these lecture

notes quickly reviews all the essential calculus for following this course. Few theorems that are repeatedly

used in the course are collected and presented with an outline of their proofs. Chapter 2 introduces the concept of errors. One may be surprised to see errors at the initial stage of the course when no methods are introduced. Of course, thereare two types of errors involved in a method, namely,

1. the error involved in approximating a problem and

2. the error due to computation.

8 ContentsThe first type of error is purely mathematical and often known astruncation error. The second one is

due to the floating-point approximation of a number. This error is committed by computer due to their

limited memory capacity. For instance, the number 1/3=0.3333... hasinfinitely many digits and since a

computer can deal with a number with finite number of digits, this number has to be approximated to

the number 0.333...3 with finite number of digits (depending on the memory capacity of the computer).

Such an approximation is called thefloating-point approximation. Chapter 2 is devoted mainly to the floating-point error and related concepts. Devising methods to solve linear systems and computation of eigenvalues and eigen vectors are the

subject of the chapter 2. In this chapter, we discuss direct methods which gives exact solution to the

systems mathematically. However, when we implement these direct methods on a computer we will get

an approximate solution as the computed solution involves floating-point error. The chapter then discuss

some iterative methods for solving linear systems. After a brief discussion of matrix analysis, the chapter

ends with power method for computing a eigenvalue and the corresponding eigen vector for a given matrix.

Not all eivenvalues can be computed using this method and also not allmatrices can be applicable to this method. Gershgorin"s theorem may be used to decide whether power method can be used for a given matrix. We state this theorem without proof and discuss its application to power method. Chapter 4 introduces various iterative methods for a nonlinear equation and their convergence anal-

ysis. The methods are further extended to system of nonlinear equations. Unconditioned optimization is

discussed at the end of the chapter. Interpolation by polynomials,data fitting and least-square approx-

imation are the subject of Chapter 5. Chapter 6 introduced numerical differentiation and integration.

These notes end with some basic methods for solving ordinary differential equations.

1Mathematical PreliminariesThis chapter reviews some of the results from calculus that are frequently used in this course. Only

definitions and important theorems with outline of a proof are provided. However, the readers are assumed

to be familiar with a first course in calculus. Section 1 defines continuity of a function and proves intermediate value theorem. This theorem plays

a basic role in finding initial guess in iterative methods for solving nonlinear equations in chapter 3.

Derivative of a function, Rolle"s theorem and the mean-value theorem for derivatives in provided in section 2. The mean-value theorem for integration is discussed in section 3. These two theorems are

crucially used in deriving truncation error for numerical methods. Finally, Taylor"s theorem is discussed

in section 4, which is essential for derivation and error analysis of almost all numerical methods discussed

in this course. Throughout this chapter, we use the notation [] for a closed interval and () for an open interval, whereandare some finite real numbers such that .

1.1 Continuity of a Function

Definition 1.1 (Continuity).

A function:RRis said to becontinuousat a point0Rif lim xx0() =(0)(1.1) In other words, for any given 0, there exists a 0such that ()?(0) whenever?0 (1.2) y x 0

Fig. 1.1.=2

Example 1.2.Consider the function() =2. Clearly,() =220when0. Thus, this function is continuous. Let us now check the condition (1.2). We have, ()?(0)=2?20=+0 ?0=?0+ 20 ?0 ?0(?0+ 20)

10 1 Mathematical PreliminariesFor any given 0, choose 0 ?0+

20+ 0 to get (1.2) as required. An illustration of this

example is depicted in figure 1.1. Remark 1.3.Note that thein the above example depends on0. For a continuous function, if for any given 0, thedoes not depend on0, then the function is said to beuniformly continuous.

Theorem 1.4 (Intermediate-Value Theorem).

Let()be a continuous function on the interval[]. If(1) (2)for some numberand some1,2[]with1 2, then =()for some[]

Proof:Let:=[12] :() and:= sup.

(1) Clearly, there exists a sequenceninsuch thatn. Sinceis continuous at, we have (n)(), which implies(). (2) The sequence n=+2? [12] N converges toand hence(n)(). Asn (n), and hence(). Combining the above two inequalities, we see that() =and it is clear that[], which completes the proof.

1.2 Differentiation of a Function

Definition 1.5 (Differentiation).

A function: ()Ris said to bedifferentiableat a point()if the limit lim h0(+)?() exists. In this case, the value of the limit is denoted by()and is called thederivativeofat. The functionis said to be differentiable in()if it is differentiable at every point in(). Remark 1.6.There are two other ways to define the derivative of a continuous function: ()R.

Let us list all the three equivalent definitions

() = limh0(+)?() () = limh0()?(?) () = limh0(+)?(?) 2 where(). For any fixed 0, the formulae h() :=(+)?() (1.3) h() :=()?(?) (1.4)

0h() :=(+)?(?)

2(1.5)

are called theforward difference,backward differenceandcentral differenceformulae. The ge-

ometrical interpretation of the above three formulae is shown in figure 1.2. More discussion on these

difference operators is found in chapter 6 of these notes.

1.3 Integration of a Function 11

. ..xx-h x+hForward

BackwardCentral

xy y=f(x) f' Fig. 1.2.Geometrical interpretation of difference operators

Theorem 1.7 (Rolle"s Theorem).

Let()be continuous on the bounded interval [] and differentiable on (). If() =(), then () = 0for some()

Proof:Let[] be such that

() = min() :[]and() = max() :[] If eitheroris an interior point of [], then the result follows from the problem 11. Otherwise, bothandare end points of [] and hence() =(). Thus, the maximum and the minimum values ofon [] coincide. Hence,is constant on [], and therefore,() = 0 for every().

Theorem 1.8 (Mean-Value Theorem for Derivatives).

If()is continuous on a bounded interval[](with=) and differentiable on(), then ?=()for some()

Proof:Consider: []Rdefined by

() =()?()?(?)where=()?() Then() = 0 and the choice of the constantis such that() = 0. So, Rolle"s theorem applies to, and as a result, there is() such that() = 0. This implies that() =, as desired.

1.3 Integration of a Function

Theorem 1.9 (Mean-Value Theorem for Integrals).

Let()be a non-negative or non-positive integrable function on[]. If()is continuous on[], thenb a b a ()for some[] Proof:Assume thatis non-negative on []. Then we have b a b a b a whereandare the minimum and maximum ofin the interval []. Ifb a()= 0, then we haveb a ()()= 0

12 1 Mathematical Preliminariesin which case the result is trivial. Assume the contrary and divide bothsides of the above inequality byb

a()to get where () =1 b a() b a Sinceis continuous, intermediate-value theorem tells us that there is a[] such that() =(), which proves the theorem. Whenis non-positive, replaceby?and the same argument as above proves the theorem.

1.4 Taylor"s Formula

Theorem 1.10 (Taylor"s Formula with Remainder).

If()has+ 1continuous derivatives on[]andis some point in[], then for all[] () =() +()(?) +()(?)2

2!++(n)()(?)n!+n+1()

where n+1() =1 x c (?)n(n+1)()

Proof:We prove the formula by induction.

(1) Let us first prove the formula for= 1 for which we have

2() =()?()?()(?) =

x c x c x c

The last integral may be written as

x c, where=()?()and=?. Now=() and= 1, so by the integration by parts, we have 2() = x c =xc? x c x c since= 0 when=, and= 0 when=. This completes the proof when= 1. (2) We now assume that the formula is true for someand prove it for+ 1. The Taylor"s formula for + 1 can be written as n+1() =()? () +()(?) +()(?)2

2!++(n1)()(?)n1(?1)!+(n)()(?)n!

=n()?(n)()(?)n Since, the Taylor"s formula holds for, we can use the given remainder formula forn(). Using the identity (?)n x c (?)n1 we obtain n+1() =1 (?1)! x c (?)(n1)(n)()?(n)()(?1)! x c (?)n1 1 (?1)! x c (?)(n1)[(n)()?(n)()]

The last integral may be written in the form

x c, where=(n)()?(n)() and=?(?)n Noting that= 0 when=, and that= 0 when=, we get from integration by parts

1.4 Taylor"s Formula 13

quotesdbs_dbs21.pdfusesText_27

[PDF] numerical analysis book pdf by jain and iyengar

[PDF] numerical analysis books indian authors

[PDF] numerical analysis bsc 3rd year

[PDF] numerical analysis handwritten notes pdf

[PDF] numerical analysis pdf download

[PDF] numerical analysis pdf for computer science

[PDF] numerical analysis pdf s.s sastry

[PDF] numerical analysis pdf sauer

[PDF] numerical analysis pdf solutions

[PDF] numerical analysis questions and answers pdf

[PDF] numerical mathematical analysis pdf

[PDF] numerical methods for computer science pdf

[PDF] numerical methods for engineering and science by saumyen guha pdf

[PDF] numerical methods for scientific and engineering computation 4th edition pdf

[PDF] numerical methods for solving system of nonlinear equations