[PDF] [PDF] Probability Review - Discrete Random Variables

[PDF] Discrete random variables - UConn Undergraduate Probability OER

(i) E[X + Y ] = EX + EY , (ii) E[aX] = aEX, as long as all expectations are well- defined PROOF Consider a random variable Z := X + Y which is a discrete random 

[PDF] 11 Discrete Random variables

If X is a random variable on S and g : R → R then Y = g(X) is a new random variable on X which maps S into R by Y (s) = g(X(s)) for all outcomes s ∈ S For example Y = 2X − 7 or Z = X2 are both new random variables on S Let X be a discrete random variable

[PDF] Probability Review - Discrete Random Variables

25 sept 2019 · P[X ∈ {1, 2, 3}] Random variables are usually divided into discrete and continuous, even The support SX of the discrete random variable X is more rigorous way of showing that (1 5 4) is correct is to evaluate the sums

[PDF] Discrete Random Variables and Probability Distributions

Can we show that the two equations for variance are equal? V (x) = σ 2 = E(X 2 ) 

[PDF] Chapter 2 Discrete random variables - CERMICS

Let α ∈ R We say that a discrete random variable X follows the degen- Proof Let us show that X1 + ··· + Xn ∼ B(n, p) Let k ∈ {0,··· ,n} By σ-additivity,

[PDF] RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS 11

serve as the probability distribution for a discrete random variable X if and only if it s Proof for case of finite values of X Consider the case where the random 

[PDF] 3 Discrete Random Variables - EPFL

where p is the probability that a coin shows a head Write down the sample space and the sets Ax when n = 3 What is the random variable X = I1 + ··· + In?

[PDF] Discrete Random Variables I - David Dalpiaz

Informally, a random variable is a quantity X whose value depends on some Probability mass function (p m f ) (also called a “discrete density function”, or,

[PDF] 2 Discrete Random Variables - Arizona Math

Definition 1 A discrete random variable X on a probability space (Ω, F, P) Proof The proof of the theorem is trivial First note that if replace A by its intersection 

[PDF] Discrete Random Variables, I - Illinois

Informally, a random variable is a quantity X whose value depends on some Probability mass function (p m f ) (also called a “discrete density function”, or,

[PDF] show that x is a markov chain

[PDF] show that x is a random variable

[PDF] show that [0

[PDF] show the mechanism of acid hydrolysis of ester

[PDF] show time zone cisco

[PDF] show ∞ n 2 1 n log np converges if and only if p > 1

[PDF] shredded workout plan pdf

[PDF] shredding diet

[PDF] shredding workout plan

[PDF] shrm furlough

[PDF] shuttle paris

[PDF] si clauses french examples

[PDF] si clauses french exercises

[PDF] si clauses french practice

[PDF] si clauses french practice pdf

Lecture1: Discrete random variables1of15Course: Introduction to Stochastic Processes

Term: Fall2019

Instructor: Gordan žitkovi´cLecture1

Discrete random variables

1.1Random Variables

A large chunk of probability is about random variables. Instead of giving a precise definition, let us just mention that arandom variablecan be thought of as an uncertain quantity (usually numerical, i.e., with values in the set of real numbersR, but not always). While it is true that we do not know with certainty what value a random variableXwill take, we usually know how to assign a number - the proba- bility - that its value will be in some

1subset ofR. For example, we might be

interested inP[X7],P[X2[2,3.1]]orP[X2 f1,2,3g]. Random variables are usually divided intodiscreteandcontinuous, even though there exist random variables which are neither discrete nor continu- ous. Those can be safely neglected for the purposes of this course, but they play an important role in many areas of probability and statistics.

1.2Discrete random variables

Before we define discrete random variables, we need some vocabulary.Definition1.2.1.Given a setB, we say that the random variableXis

B-valuedifP[X2B] =1.In words,XisB-valued if we know for a fact thatXwill never take a value outside ofB.Definition1.2.2.A random variable is said to bediscreteif there exists a setSsuch thatSis either finite or countableaandXisS-valued.a Countable means that its elements can be enumerated by the natural numbers. The only (infinite) countable sets we will need areN=f1,2,...,gorN0=f0,1,2,...g.1

We will not worry about measurability and similar subtleties in this class.Last Updated:September25,2019

Lecture1: Discrete random variables2of15Definition1.2.3.ThesupportSXof the discrete random variableXis

the smallest setSsuch thatXisS-valued.Example1.2.4.A die is thrown and the number obtained is recorded and

denoted by X.The possible values ofXaref1,2,3,4,5,6gand each happens with probability

1/6, soXis certainlyS-valued. SinceSis

finite,Xis discrete. One still needs to argue thatSis the supportSXofX. The alternative would be thatSXis a proper subset ofS, i.e., that there are redundand elements inS. This is not the case since all elements inSare "impor- tant", i.e., happen with positive probability. If we remove anything fromS, we are omitting a possible value forX. On the other hand, it is certainly true thatXalways takes its values in the finite setS0=f1,2,3,4,5,6,7g, i.e., thatXisS0-valued. One has to be careful with the terminology here: it is correct to say thatXis anS0-valued (or evenN-valued) random variable, even though it only

takes the values 1,2,...,6 with positive probabilities.Discrete random variables are very nice due to the following fact: in or-

der to be able to compute any conceivable probability involving a discrete random variableX, it is enough to know how to compute the probabili- tiesP[X=x], for allx2 S. Indeed, if you are interested in figuring out whatP[X2B]is, for some setBR(e.g.,B=f5,6,7g,B= [3,6], or B= [2,¥)), we simply pick allx2 SXwhich are also inBand sum their probabilities. In mathematical notation, we have

P[X2B] =å

x2SX\BP[X=x]. (1.2.1)Definition1.2.5.Theprobability mass function (pmf)of a discrete random variableXis a functionpXdefined on the supportSXofXby p X(x) =P[X=x],x2 SX.In practice, we usually present the pmfpXin the form of a table (called thedistribution table) as Xxx

1x2x3...p

X(x)p

1p2p3...

or, simply, Xx

1x2x3...p

1p2p3...,Last Updated:September25,2019

Lecture1: Discrete random variables3of15where the top row lists all the elementsxof the supportSXofX, and the

bottom row lists their probabilitiespX(x) =P[X=x]. It is easy to see that the functionpXhas the following properties:

1.pX(x)2[0,1]for allx, and

2.åx2SXpX(x) =1.

Here is a first round of examples of discrete random variables and their supports.Example1.2.6.

1.A fair (unbiased) coin is tossed and the value observed is denoted by X.

Since the only possible valuesXcan take areHorT, and the set S=fH,Tgis clearly finite,Xis a discrete random variable. Its distribution is given by the following table: xH T p X(x)1 /21/2 BothHandTare possible (happen with probability1/2), so no smaller setSwill have the property thatP[X2 S] =1. Conse- quently, the supportSXofXisS=fH,Tg.

2.A die is thrown and the number obtained is recorded and denoted by X.

The possible values ofXaref1,2,3,4,5,6gand each happens with probability

1/6, soXis discrete with supportSX. Its distribution is

given by the table x1 2 3 4 5 6 p X(x)1 /61/61/61/61/61/6

3.A fair coin is thrown repeatedly until the first H is observed; the number

of Ts observed before that is denoted by X.In this case we know that Xcan take any of the valuesN0=f0,1,2,...gand that there is no finite upper bound for it. Nevertheless, we know thatXcannot take values that are not non-negative integers. Therefore,XisN0- valued and, in fact,SX=N0is its support. Indeed, we have

P[X=x] =2x1, forx2N0, i.e.,

Xx0 1 2 ...

p X(x)1 /21/41/8...

4.A card is drawn randomly form a standard deck, and the result is de-

noted by X.This example is similar to the2., above, sinceXtakes one of finitely many values, and all values are equally likely. TheLast Updated:September25,2019

Lecture1: Discrete random variables4of15difference is that the result is not a number anymore. The set

Sof all possible values can be represented as the set of all pairs like(,7), where the first entry denotes the picked card"s suit (in f~,,|,}g), and the second is a number between 1 and 13. It is, of course, possible to use different conventions and use the set f2,3,...,9,10,J,Q,K,Agfor the second component. The point is that the valuesXtakes are not numbers.1.3Events and Bernoulli random variables Random variablesXwhich can only take one of two values 0,1, i.e., for whichSX f0,1g, are calledindicatorsorBernoulli random variablesand are very useful in probability and statistics (and elsewhere). The name comes from the fact that you should think of such variables as signal lights; ifX=1 an event of interest has happened, and ifX=0 it has not happened. In other words,X indicatesthe occurence of an event. One reason the Bernoulli random variables are so useful is that they let us manipulateeventswithout ever leaving the language of random variables. Here is an example:Example1.3.1.Suppose that two dice are thrown so thatX1andX2are the numbers obtained (bothX1andX2are discrete random variables withSX1=SX2=f1,2,3,4,5,6g). If we are interested in the probabil- ity that their sum is at least 9, we proceed as follows. We define the random variableW- the sum ofX1andX2- byW=X1+X2. An- other random variable, let us call itX, is a Bernoulli random variable defined by X=( 1,W9,

0,W<9.

With such a set-up,Xsignals whether the event of interest has hap- pened, and we can state our original problem in terms ofX, namely "ComputeP[X=1]!".This example is, admittedly, a little contrived. The point, however, is that anything can be phrased in terms of random variables; thus, if you know how to work with random variables, i.e., know how to compute their distributions, you can solve any problem in probability that comes your way. Another reason Bernoulli random variables are useful is the fact that we can do arithmetic with them.Last Updated:September25,2019

Lecture1: Discrete random variables5of15Example1.3.2.70 coins are tossed and their outcomes are denoted by

W

1,W2, ...,W70. AllWiare random variables with values infH,Tg

(and therefore not Bernoulli random variables), but they can be easily recodedinto Bernoulli random variables as follows: X i=(

1, ifWi=H,

0, ifWi=T.

Once you have the "dictionary"f1$H,0$Tg, random variablesXi andWicarry exactly the same information. The advantage of usingXi is that the random variable

N=70å

i=1X i, which takes values inSN=f0,1,2,...,70gcounts the number of heads amongW1,...,W70. Similarly, the random variable

M=X1X2 X70

is a Bernoulli random variable itself. What event does it indicate?1.4Some widely used discrete random variables

The distribution of a random variable is sometimes defined as "the collection of all possible probabilities associated to it". This sounds a bit abstract, and, at least in the discrete case, obscures the practical significance of this impor- tant concept. We have learned that for discrete variables the knowledge of the pmf or the distribution table (such as the one in part1.,2. or3. of Example

1.2.6) amounts to the knowledge of the whole distribution. It turns out that

many random variables in widely different contexts come with the same (or similar) distribution tables, and that some of those appear so often that they deserve to be named (so that we don"t have to write the distribution table every time). The following example lists some of those,named, distribution. There are many others, but we will not need them in these notes.Example1.4.1.

1.Bernoulli distribution.We have already encountered this distri-

bution in our discussion of indicator random variables above. It is characterized by the distribution table of the form0 1

1p p, (1.4.1)Last Updated:September25,2019

Lecture1: Discrete random variables6of15wherepcan be any number in(0,1). Strictly speaking, each value

ofpdefines a different distribution, so it would be more correct to speak of aparametric familyof distributions, withp2(0,1)being theparameter. In order not to write down the table (1.4.1) every time, we also use the notationXB(p). For example, the Bernoulli random variable which takes the value 1 when a fair coin fallsHand 0 when it falls

Thas aB(1/2)-distribution.

An experiment (random occurrence) which can end in two possible ways (usually calledsuccessandfailure, even though those names should not always be taken literally) is often called aBernoulli trial. If we "encode" success as 1 and failure by 0, each Bernoulli trial gives rise to a Bernoulli random variable.

2.Binomial distribution.A random variable whose distribution ta-

ble looks like this0 1 ...(n1)nq n(n

1)pqn1...(n

n1)qpn1pn for somen2N,p2(0,1)andq=1p, is called thebinomial distribution, usually denoted byb(n,p). Remember that thebino- mial coefficient (n k)is given by n k =n!k!(nk)!wheren!=n(n1)(n2)...21. Binomial distribution(s) form a parametric family with two param- etersn2Nandp2(0,1), and each pair(n,p)corresponds to a different binomial distribution. 012n yFigure1.The probability mass function (pmf) of a typical binomial distribution.Last Updated:September25,2019

Lecture1: Discrete random variables7of15Recall that the binomial distribution arises as the "number of suc-

cesses innindependent Bernoulli trials", i.e., it counts the number ofHinnindependent tosses of a biased coin whose probability of Hisp.

3.Geometric distribution.The geometric distribution is similar to

the binomial in that it is built out of the sequence of "successes" and "failures" in independent, repeated Bernoulli trials. The dif- ference is that the number of trials is no longer fixed (i.e.,=n), but we keep tossing until we get our first success. Since the trials are independent, if the probability of success in each trial isp2(0,1), the probability that it will take exactlykfailures before the first success isqkp, whereq=1p. Therefore, so the geometric distri- bution - denoted byg(p)- comes with the following table0 1 2 3 ... p qp q

2p q3p....

012 yFigure2.The probability mass function (pmf) of a typical geometric distribution.Last Updated:September25,2019 Lecture1: Discrete random variables8of15Caveat:When defining the geometric distribution, some people count the number of trials to the first success, i.e., add the final success into the count. This shifts everything by 1 and leads to a distribution with supportN(and not N

0). While this is no big deal, this ambiguity tends to be

confusing at times and leads to bugs in software. For us, the geometric distribution will always start from 0. The distri- bution which counts the final success will be referred to as theshifted geometric distribution, but we"ll try to avoid it altogether.4.Poisson distribution.This is also a family of distributions, param- eterized by a single parameterl>0, and denoted byP(l). Its support isN0and the distribution table is given by0 1 2 3 4 ... e lellell22 ell33! ell44!

The closed form for the pmf is

p

X(x) =ellxx!,x2N.

The Poisson distribution arises as a limit whenn!¥andp!0 whilenplin the Binomial distribution. 012 yFigure3.The probability mass function (pmf) of a typical Poisson distribution withl>1.Last Updated:September25,2019 Lecture1: Discrete random variables9of151.5Expectations and standard deviations Expectations and standard deviations provide summaries of numerical ran- dom variables - they give us some information about them without over- whelming us with the entire distribution table. The expectation can be thought of as acenterof the distribution, while the standard deviation gives you an idea about itsspread2.Definition1.5.1.For a discrete random variableXwith supportSX

R, we define theexpectationE[X]ofXby

E[X] =å

x2SXx p

X(x), (1.5.1)

if the (possibly) infinite sum

åx2Sx pX(x)absolutely converges, i.e., as

long as x2SXjxjpX(x)<¥.(1.5.2) When the sum in (1.5.2) above diverges (i.e., takes the value+¥), we

say that the expectation ofXis not defined.Perhaps the most important property of the expectation is its linearity:

Theorem1.5.2.IfE[X]andE[Y]are both defined then so isE[aX+bY], for any two constantsa,b. Moreover, E[aX+bY] =aE[X] +bE[Y].In order to define the standard deviation, we first need to define the vari- ance. Like the expectation, the variance may or may not be defined (depend- ing on whether the sums used to compute it converge absolutely or not). Since we will be working only with distributions for which the existence of

expectation(s) is never a problem, we do not mention this issue in the sequel.Definition1.5.3.Thevarianceof the random variableXis

Var[X] =E[(XmX)2] =å

x2SX(xmX)2pX(x)wheremX=E[X].

Thestandard deviationofXis

sd[X] =qVar[X].2 this should be taken with a grain of salt. After all, what exactly do we mean by acenteror aspreadof a distribution?Last Updated:September25,2019

Lecture1: Discrete random variables10of15The fundamental properties of the variance/standard deviation are given

in the following theorem

3:Theorem1.5.4.Suppose that X and Y are random variables and thatais a

constant. Then

1.Var[aX] =a2Var[X], and

2.if, additionally ,X and X ar eindependent, then

Var[X+Y] =Var[X] +Var[Y].Caveat:These properties are not the same as the properties of the ex- pectation. First of all the constant comes out of the variancewith a square, and second, the variance of the sum is the sum of the indi- vidual variances only if additional assumptions, such as theindepen-

dencebetween the two variables, are imposed.Finally, here is a very useful alternative formula for the variance of a

random variable:Proposition1.5.5.Var[X] =E[X2](E[X])2.Let us compute expectations and variances/standard deviations for our

most important examples.Example1.5.6.

1.Bernoulli distribution.LetXB(p)be a Bernoulli random vari-

able with parameterp. Then (rememberqis a shortcut for 1p)

E[X] =0q+1p=p.

Using (1.5.5), we get

Var[X] =E[X2](E[X])2=02q+12pp2

=pp2=p(1p) =pq, and, so, sd[X] =ppq.3 we will talk about independence in detail in the next lecture. An intuitive understanding should be fine for now.Last Updated:September25,2019 Lecture1: Discrete random variables11of152.Binomial distribution.Moving on to the binomial,Xb(n,p), we could either use the formula (1.5.1) and try to evaluate the sum

E[X] =nå

k=0kn k p kqnk, or use some of the properties of the expectation of Theorem1.5.2. To do the latter, we remember that the distribution of a binomial is the same as the distribution of a sum ofn(independent) Bernoul- lies. So if we writeX=X1++Xn, and eachX1...Xnhas the

B(p)distribution, Theorem1.5.2yields

E[X] =E[X1] +E[X2] ++E[Xn] =np. (1.5.3)

A similar simplification can be achieved in the computation of the variance, too. While it was unimportant in thatX1,...,Xnare in- dependent in (1.5.3), it is crucial for Theorem1.5.4:

Var[X] =Var[X1] ++Var[Xn] =npq,

and, so, sd[X] =pnpq.

3.Geometric distribution.The trick from2. above cannot be applied

to the geometric random variables. If nothing else, this is because Theorem1.5.2can only be applied to a given (fixed, nonrandom) numbernof random variables. We can still use the definition (1.5.1) and evaluate an infinite sum:

E[X] =¥å

k=0kpqk. Instead of doing that, let us proceed somewhat informally and note that we can think of a geometric random variable as follows: With probabilitypour first throw is a success andX=

0. With probabilityqour first throw is a failure and we

restart the experiment on the second throw, making sure to add the first failure to the count.

Therefore,

E[X] =p0+q(1+E[X]),

and, so,E[X] =q/p.

Similar reasoning can be applied to obtain

E[X2] =p0+qE[(1+X)2] =q+2qE[X] +qE[X2]

=q+2q2/p+qE[X2], which yields Var[X] =E[X2](E[X])2=q/p2and sd[X] =pq /p.Last Updated:September25,2019

Lecture1: Discrete random variables12of154.Poisson distribution.We know that the Poisson distribution arises

as a limit of binomial distributions whenn!¥,p!0 and npl. We can expect, therefore, that its expectation and vari- ance should behave accordingly, i.e., forXP(l), we have

E[X] =land Var[X] =l. (1.5.4)

The reasoning behind Var[X] =luses the formula Var[X] =npq whenXb(n,p)and plugs inq1, sinceq=1pandp!0. A more rigorous way of showing that (1.5.4) is correct is to evaluate the sums

E[X] =¥å

k=0kp

X(k) =¥å

k=0kellk/k! and

E[X2] =¥å

k=0k2pX(k) =¥å k=0k2ellk/k!. and use Proposition1.5.5. The sums can be evaluated explicitly, but since the focus of these notes is not on evaluation of infinite sums, so we skip the details.1.6Problems Problem1.6.1.Two people are picked at random from a group of50and given $10each. After that, independently of what happened before, three people are picked from the same group - one or more people could have been picked both times - and given $10each. What is the probability that at least one person received $20? Problem1.6.2.A die is rolled5times; let the obtained numbers be given by Y

1,...,Y5. Use counting to compute the probability that

1.all Y1,...,Y5are even?

2.at most 4ofY1,...,Y5are odd?

3.the v aluesof Y1,...,Y5are all different from each other?

Problem1.6.3.Identify the supports of the following random variables:

1.Y+1, whereYB(p)(Bernoulli),

2.Y2, whereYb(n,p)(binomial),Last Updated:September25,2019

Lecture1: Discrete random variables13of153.Y5, whereYg(p)(geometric),

4.2 Y, whereYP(l)(Poisson).

Problem1.6.4.LetYdenote the number of tosses of a fair die until the first

6 is obtained (if we get a 6 on the first try,Y=0). The supportSYofYis

(a)f0,1,2,3,4,...g (b)f1,2,3,4,5,6g (c)f16 ,16 ,16 ,16 ,16 g (d)f16 ,56 16 ,56 216
,56 316
,...g (e) none of the abo ve Problem1.6.5.The probability that Janet makes a free throw is 0.6. What is the probability that she will make at least 16 out of 23 (independent) throws? Write down the answer as a sum - no need to evaluate it. Problem1.6.6.Three unbiased and independent coins are tossed. LetY1be the total number of heads on the first two coins, and letYbe the random variable which is equal toY1if the third coin comes upheadsandY1if it comes uptails. Compute Var[Y]. Problem1.6.7.A die is thrown and a coin is tossed independently of it. Let Ybe the random variable which is equal to the number on the die in case the coin comes upheadsand twice the number on the die if it comes uptails.

1.What the support of SYofY? What is its distribution (pmf)?

2.Compute E[Y]and Var[Y].

Problem1.6.8.npeople vote in a general election, with only two candidates running. The vote of personiis denoted byYiand it can take values 0 and 1, depending which candidate they voted for (we encode one of them as 0 and the other as 1). We assume that votes are independent of each other and that each person votes for candidate 1 with probabilityp. If the total number of votes for candidate 1 is denoted byY, then (a)Yis a geometric random variable (b)Y2is a binomial random variableLast Updated:September25,2019 Lecture1: Discrete random variables14of15(c)Yis uniform onf0,1,...,ng (d)

V ar[Y]E[Y]

(e) none of the abo ve Problem1.6.9.A discrete random variableYis said to have adiscrete uni- form distributiononf0,1,2,...,ng, denoted byYu(n)if its distribution table looks like this:0 1 2 ...n1n+11n+11n+1...1n+1. Compute the expectation and the variance ofu(n). You may use the fol- lowing identities: 1+2++n=12 n(n+1)and 12+22++n2= 16 n(n+1)(2n+1). Problem1.6.10.LetXbe a Poisson random variable with parameterl>0.

Compute the following:

1.P[X3],

2.(*) E[X3].Note:The sum you need to evaluate is quite difficult. If you don"t know

the trick, do not worry. If you know how to use symbolic-computation software such as Mathematica, feel free to use it. We will learn how to do this using generating functions later in the class. Problem1.6.11.LetXbe a geometric random variable with parameterp2 (0,1), i.e.Xg(p), and letY=2X. Write down the (first few entries in) the distribution table ofY. ComputeE[Y] =E[2X]. Problem1.6.12.LetY1andY2be uncorrelated discrete random variables such that Var[2Y1Y2] =17 and Var[Y1+2Y2] =5. Compute Var[Y1Y2].Note: Y

1andY2areuncorrelatedifE[(Y1E[Y1])(Y2E[Y2])] =0.

(Hint:What is Var[aY2+bY2]in terms of Var[Y1]and Var[Y2]whenY1 andY2are uncorrelated?) Problem1.6.13.LetY1andY2be uncorrelated random variables such that sd[Y1+Y2] =5. Then sd[Y1Y2] = (a) 1 (b) p2 (c) p3 (d) 5 (e) not enough information is given

Last Updated:September25,2019

Lecture1: Discrete random variables15of15Problem1.6.14.LetXbe a discrete random variable with the supportSX=

N, such thatP[X=n] =C1n

2, forn2N, whereCis a constant chosen so

that ånP[X=n] =1. The distribution table ofXis, therefore, given by1 2 3 ... C 11 2C12 2C13quotesdbs_dbs14.pdfusesText_20