Order statistic - Definition

In statistics, the nth order statistic of a sample is equal to the nth-smallest sample value.

For example, if the sample values are:

6, 9, 3, 8

then the second order statistic is:

x₍₂₎ = 6.

The first order statistic (or smallest order statistic) is always the minimum of the sample. For a sample of size n, the nth order statistic (or largest order statistic) is the maximum.

In conventional notation, one writes:

x₁ = 6

x₂ = 9

x₃ = 3

x₄ = 8

with no round brackets () in the subscripts, referring to the data in the order in which they appear, and:

x₍₁₎ = 3

x₍₂₎ = 6

x₍₃₎ = 8

x₍₄₎ = 9

with round brackets () in the subscripts, referring to the order statistics, i.e., the data sorted into increasing order.

Probability distribution

The kth order statistic

Let <math>X_{1}, X_{2}, \ldots, X_{n}<math> be iid continuously distributed random variables, and <math>X_{(1)}, X_{(2)}, \ldots, X_{(n)}<math> be the corresponding order statistics. Let f(x) be the probability density function and F(x) be the cumulative distribution function of X_i. Then the probability density of the k^th statistic can be found as follows.

<math>f_{X_{(k)}}(x)={d \over dx} F_{X_{(k)}}(x)

={d \over dx}P\left(X_{(k)}\leq x\right)={d \over dx}P(\mathrm{at}\ \mathrm{least}\ k\ \mathrm{of}\ \mathrm{the}\ n\ X\mathrm{s}\ \mathrm{are}\leq x)<math>

<math>={d \over dx}P(\geq k\ \mathrm{successes}\ \mathrm{in}\ n\ \mathrm{trials})={d \over dx}\sum_{j=k}^n{n \choose j}P(X_1\leq x)^j(1-P(X_1\leq x))^{n-j}<math>

<math>={d \over dx}\sum_{j=k}^n{n \choose j} F(x)^j (1-F(x))^{n-j}<math>

<math>=\sum_{j=k}^n{n \choose j}

\left(jF(x)^{j-1}f(x)(1-F(x))^{n-j} +F(x)^j (n-j)(1-F(x))^{n-j-1}(-f(x))\right)<math>

<math>=\sum_{j=k}^n\left(n{n-1 \choose j-1}F(x)^{j-1}(1-F(x))^{n-j} - n{n-1 \choose j} F(x)^j(1-F(x))^{n-j-1} \right)f(x)<math>

<math>=nf(x)\left(\sum_{j=k-1}^{n-1} {n-1 \choose j}

F(x)^j (1-F(x))^{(n-1)-j} - \sum_{j=k}^n {n-1 \choose j} F(x)^j (1-F(x))^{(n-1)-j}\right)<math>

and the sum above telescopes, so that all terms cancel except the first and the last:

<math>=nf(x)\left({n-1 \choose k-1} F(x)^{k-1} (1-F(x))^{(n-1)-(k-1)}

- \underbrace{{n-1 \choose n} F(x)^n (1-F(x))^{(n-1)-n}}\right)<math>

and the term over the underbrace is zero, so:

<math>=nf(x){n-1 \choose k-1} F(x)^{k-1} (1-F(x))^{(n-1)-(k-1)}<math>

<math>={n! \over (k-1)!(n-k)!} F(x)^{k-1} (1-F(x))^{n-k} f(x).<math>

Joint densities

Similarly, for i < j, the joint probability density function of two order statistics X_i and X_j can be shown to be

<math>f_{X_{(i)},X_{(j)}}(u,v) = \frac{n!}{(i-1)!\,(j-i-1)!\,(n-j)!}F(u)^{i-1}(F(v)-F(u))^{j-i-1}(1-F(v))^{n-j}f(u)f(v)<math>

The joint distribution of vector of order statistics is given by

<math>f_{X_{(1)},X_{(2)},\ldots,X_{(n)}}(z_{1},z_{2},\ldots,z_{n}) = n! \prod_{i=1}^{n} f(z_{i}).\,<math>

See also

• Quantile
• Median
• Rankit
• Box plot
• Fisher-Tippett distribution

External links

• PlanetMath: order statistics (http://planetmath.org/encyclopedia/OrderStatistics.html) Retrieved Feb 02,2005
• Eric W. Weisstein. "Order Statistic." From MathWorld--A Wolfram Web Resource. MathWorld Order Statistic (http://mathworld.wolfram.com/OrderStatistic.html) Retrieved Feb 02,2005
• Dr. Susan Holmes Order Statistics (http://www-stat.stanford.edu/~susan/courses/s116/node79.html) Retrieved Feb 02,2005