Lecture15Slides.Rmd

---
title: "Stat 201 - Lecture 15"
author: "Prof. Heggeseth"
date: "November 5, 2015"
output: slidy_presentation
---

```{r global_options, include=FALSE}
knitr::opts_chunk$set(dev='png',fig.path='Figs/',echo=TRUE,message=FALSE, warning=FALSE,message=FALSE,tidy=TRUE,fig.width = 6,fig.height=6)
```

----------

#Summary of Discrete RV

**Bernoulli RV** ($X = 0$ or $1$)
\[P(X = x) = p^x(1-p)^{1-x}\]
\[E(X) = p\]
\[Var(X) = p(1-p) \]

*Note: We proved this in class using the definitions.*


**Binomial RV** (sum of independent Bernoulli RV's)
\[P(X = x) = \frac{n!}{x!(n-x)!}p^x(1-p)^{n-x}\]
\[E(X) = np\]
\[Var(X) = np(1-p) \]

*Note: You can prove $E(X) = np$ and $Var(X) = np(1-p)$ if you let $X = X_1+\cdots+X_n$ where $X_i$ is Bernoulli.*

**Poisson RV**
\[P(X = x) = \frac{e^{-\lambda}\lambda^x}{x!}\]
\[E(X) = \lambda\]
\[Var(X) = \lambda \]

*Note: The proof for $E(X) = \lambda$ (using the definition) was in last lecture's slides.*

--------------

#Poisson RV's

Besides being a good approximation for the Binomial, the Poisson model is quite useful...

- if we have two possible outcomes
- if we know the average number of successes in a specific region (not fixed number of trials)
- if the number of successes in two disjoint regions are independent
- if the probability is proportion to the size of region
- if the probability in an extremely small region is virtually zero

##Examples

- X: number of deaths by horse kicking in the Prussian army
- X: number of car accidents in Williamstown this month
- X: number of typing errors on a page
- X: number of website updates within an hour
- X: number of fish caught in an hour


--------------

#Continuous RV's

For continuous random variables $X$, 

we define the probability model using a **culmulative distribution function** (cdf), 
\[F(x) = P(X\leq x)\]
*(it is always notated with a capital letter $F$ or $G$ or $H$)*.

and a **probability density function** (pdf), $f(x)\geq 0$ such that 
\[P(a\leq X \leq b) = \int^b_a f(x)dx\]
*(it is always notated with a small letter $f$ or $g$ or $h$)* and 
\[P(S) = P(-\infty\leq X\leq \infty) = \int^\infty_{-\infty}f(x)dx = 1\]

Thus, $F(x) = \int^x_{-\infty} f(y)dy$. 

--------------

##Moments
Let $X$ be a continuous RV with pdf $f(x)$,

\[E(X)= \int^\infty_{-\infty} xf(x)dx \]
\[E(g(X))= \int^\infty_{-\infty} g(x)f(x)dx \]


\[Var(X)= E((X-E(X))^2)=E(X^2) - [E(X)]^2 \]
\[Cov(X,Y)= E((X-E(X))(Y-E(Y))) = E(XY) - E(X)E(Y) \]
\[Cor(X,Y)= \frac{Cov(X,Y)}{SD(X)SD(Y)} \]

Note: If $X$ and $Y$ are independent, then $Cov(X,Y) = 0$. The reverse is not true. 

--------------

##Examples

**Uniform Model**

For a Uniform RV $X$ that takes values between $a$ and $b$, the pdf is
\[f(x) = \begin{cases}
\frac{1}{b-a}  \text{ if } a\leq x \leq b\\
0 \text{ otherwise}
\end{cases}\]

```{r}
a = 2
b = 6
f = function(x){
  ifelse(x>a & x<b, 1/(b-a),0)
}
x = seq(-1,10,by=.1)
plot(x,f(x),type='l')
```

**How could we show that this is a legitimate pdf?**

**What is the $P(X\leq 4)$?**

**Moments**

\[E(X) = \int^b_a x\frac{1}{b-a}dx\] 
\[= \frac{1}{b-a} [\frac{1}{2}x^2]^b_a\]
\[=  \frac{b^2-a^2}{2(b-a)} \]
\[= \frac{(b+a)(b-a)}{2(b-a)}\]
\[= \frac{1}{2}(b+a) \]

\[E(X^2) = \int^b_a x^2\frac{1}{b-a}dx \]
\[= \frac{1}{b-a} [\frac{1}{3}x^3]^b_a \]
\[=  \frac{b^3-a^3}{3(b-a)}\]
\[=  \frac{(b-a)(a^2+ab+b^2)}{3(b-a)}\]

\[Var(X) = E(X^2) - E(X)^2\]
\[=\frac{(b-a)(a^2+ab+b^2)}{3(b-a)} - \frac{(b+a)^2}{4} \]
\[=  \frac{(a^2+ab+b^2)*4}{3*4} - \frac{3*(b+a)^2}{3*4}\]
\[= \frac{(a^2+ab+b^2)*4 - 3*(b+a)^2}{3*4}\]
\[= \frac{(a^2+ab+b^2)*4 - 3*(b^2+2ab+a^2)}{3*4}\]
\[= \frac{b^2+a^2 - 2ab}{3*4}\]
\[= \frac{(b-a)^2}{12}\]



--------------

##Examples

**Normal Model**

For $X$ such that $E(X) = \mu$ and $SD(X) = \sigma$, if it is a Normal random variable, then the pdf is
\[f(x) =  \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}\]

```{r}
x = seq(-10,10,.01)
f = dnorm(x,mean = 0, sd = 1)
plot(x,f,type='l')
```

$P(-1\leq X \leq 1) = F(1) - F(-1) = 0.68$

```{r}
pnorm(1) - pnorm(-1) #pnorm is the cdf
```

$P(-2\leq X \leq 2) = F(2) - F(-2) = 0.95$

```{r}
pnorm(2) - pnorm(-2)
```

$P(-3\leq X \leq 3) = F(3) - F(-3) = 0.997$

```{r}
pnorm(3) - pnorm(-3)
```


--------------

#Normal Approximation of Binomial

Let $X$ be a Binomial Random Variable and $Y$ be a Normal Random Variable.

As $n\rightarrow \infty$ ($p$ is fixed), the $P(X = x) \approx P(x-0.5 \leq Y \leq x+0.5)$. 

*Note: adding and subtracting 0.5 is the continuity correction*

```{r}
n = 1000
p = 0.2
barplot(dbinom(100:300,size = n, p = p),ylab='Probability',main='n = 1000, p = 0.2')
```

If $n=1000$ and $p=0.2$, let's compare $P(X=200)$ and $P(199.5\leq Y\leq 200.5)$.
```{r}
dbinom(200,size = n, p = p)
pnorm(200.5,mean = n*p, sd = sqrt(n*p*(1-p))) - pnorm(199.5,mean = n*p, sd = sqrt(n*p*(1-p)))
```

If $n=1000$ and $p=0.2$, let's compare $P(200\leq X\leq 210)$ and $P(199.5\leq Y\leq 210.5)$.

```{r}
sum(dbinom(200:210,size = n, p = p))
pnorm(210.5,mean = n*p, sd = sqrt(n*p*(1-p))) - pnorm(199.5,mean = n*p, sd = sqrt(n*p*(1-p)))
```

How big does $n$ have to be for the Normal approximation to be appropriate? 

Rule of Thumb: $np \geq 10$ and $n(1-p)\geq 10$ because that makes sure that $E(X)-0>3SD(X)$ (mean is at least 3 SD's from 0).

For $p=0.2$, that means that $n\geq 50$.

```{r}
n = 50
p = 0.2
barplot(dbinom(0:n,size = n, p = p),names.arg=0:n,ylab='Probability',main='n = 50, p = 0.2')
```


--------------

#Means

Say we have a sequence of independent and identically distributed (iid) random variables, $X_1,...,X_n$, *(I don't know what their probability model is but $E(X_i) = \mu$ and $Var(X_i) = \sigma^2$)*

\[E(\frac{1}{n}\sum_{i=1}^n X_i) = ?\]

\[Var(\frac{1}{n}\sum_{i=1}^n X_i) = ?\]

What is the distribution (probability model) of $Y = \frac{1}{n}\sum_{i=1}^n X_i$?