1. Probability

• $Var(X)=E(X-E(X){)}^2=E(X^2)-E(X{)}^2=M2-M1$ (moments)

• $Cov(X,Y)=E((X-E(X))(Y-E(Y)))=E(XY)-E(X)E(Y)$
• $Var(X)=Cov(X,X)$

• Standard Deviation = $\sqrt{Var(x)}$

• $P(A\cup B)=P(A)+P(B)-P(A\cap B)$

• Marginal distribution: $P(X=x)={\sum }_{y}P(X=x,Y=y)$

• If A and B are independent events: $P(A|B)=P(A)$ and $P(A\cap B)=P(A).P(B)$

  • If A and B are independent: $Cov(A,B)=0$

• A and B are conditionally independent if: $P((A\cap B)/C)=P(A|C).P(B|C)$

Conditional Probability

$P(A|B)=\frac{P(A\cap B)}{P(B)}\text{ or }\frac{P(A,B)}{P(B)}$

Chain Rule

$P(x_1,x_2,\dots ,x_n)=P(x_1)P(x_2|x_1)P(x_3|x_1,x_2)\dots P(x_n|x_1,...,x_{n-1})$

Proof

Bayes’ Formula:

$P(A|B)=\frac{P(B|A)P(A)}{P(B)}$

$P(A|B)=\frac{P(A\cap B)}{P(B)}\text{ or }P(A\cap B)=P(A|B).P(B)=P(B|A).P(A)$

In other words:

$\text{posterior}=\frac{\text{likelihood}\ast \text{prior}}{\text{evidence}}$

Law of Total Probability

$P(A)=P(A|B1).P(B1)+P(A|B2).P(B2)+...+P(A|Bn).P(Bn)$

or,

$p(x)={\sum }_1^{c}p(x|y_i)P(y_i)$

Counting

• Permutation: $P_{k,n}=\frac{n!}{(n-k)!}$
• Combination: $C_{k,n}=\frac{P_{k,n}}{k!}=\frac{n!}{k!(n-k)!}$

ToDo:

• Distributions
• Maximum Likelihood Estimator
• Maximum a posteriori probability (MAP) estimator
• The Central Limit Theorem