Expectation

Expectation#

Definition#

Definition 84 (Expectation)

Let \(\P\) be a probability function defined over the probability space \(\pspace\).

Let \(X\) be a discrete random variable with \(\S = \lset \xi_1, \xi_2, \ldots \rset\).

Then the expectation of \(X\) is defined as:

\[ \expectation(X) = \sum_{x \in X(\S)} x \cdot \P \lsq X = x \rsq \]

Existence of Expectation#

Theorem 20 (Existence of Expectation)

Let \(\P\) be a probability function defined over the probability space \(\pspace\).

A discrete random variable \(X\) with \(\S = \lset \xi_1, \xi_2, \ldots \rset\) has an expectation if and only if it is absolutely summable.

That is,

\[ \expectation \lsq \lvert X \rvert \rsq \overset{\text{def}}{=} \sum_{x \in X(\S)} \lvert x \rvert \cdot \P \lsq X = x \rsq < \infty \]

Properties of Expectation#

Let \(\P\) be a probability function defined over the probability space \(\pspace\).

Let \(X\) be a discrete random variable with \(\S = \lset \xi_1, \xi_2, \ldots \rset\).

Then the expectation of \(X\) has the following properties:

Property 1 (The Law of The Unconscious Statistician)

For any function \(g\),

\[ \expectation \lsq g(X) \rsq = \sum_{x \in X(\S)} g(x) \cdot \P \lsq X = x \rsq \]

This is not a trivial result, proof can be found here.

Property 2 (Linearity)

For any constants \(a\) and \(b\),

\[ \expectation \lsq aX + b \rsq = a \cdot \expectation(X) + b \]

Property 3 (Scaling)

For any constant \(c\),

\[ \expectation \lsq cX \rsq = c \cdot \expectation(X) \]

Property 4 (DC Shift)

For any constant \(c\),

\[ \expectation \lsq X + c \rsq = \expectation(X) \]

Property 5 (Stronger Linearity)

It follows that for any random variables \(X_1\), \(X_2\), …, \(X_n\),

\[ \expectation \lsq \sum_{i=1}^n a_i X_i \rsq = \sum_{i=1}^n a_i \cdot \expectation \lsq X_i \rsq \]

Concept#

Concept

  • Expectation is a measure of the mean value of a random variable and is deterministic. It is also synonymous with the population mean.

  • Average is a measure of the average value of a random sample from the true population and is random.

  • Average of a random sample is a random variable and as sample size increases, the average of a random sample converges to the population mean.

References and Further Readings#

  • Pishro-Nik, Hossein. “Chapter 3.2.3. Functions of Random Variables.” In Introduction to Probability, Statistics, and Random Processes, 199–201. Kappa Research, 2014.

  • Chan, Stanley H. “Chapter 3.4. Expectation.” In Introduction to Probability for Data Science, 125-133. Ann Arbor, Michigan: Michigan Publishing Services, 2021.

Contents