Empirical process
In probability theory, an empirical process is a stochastic process that characterizes the deviation of the empirical distribution function from its expectation. In mean field theory, limit theorems (as the number of objects becomes large) are considered and generalise the central limit theorem for empirical measures. Applications of the theory of empirical processes arise in non-parametric statistics.[1]
Definition
For X1, X2, ... Xn independent and identically-distributed random variables in R with common cumulative distribution function F(x), the empirical distribution function is defined by
where IC is the indicator function of the set C.
For every (fixed) x, Fn(x) is a sequence of random variables which converge to F(x) almost surely by the strong law of large numbers. That is, Fn converges to F pointwise. Glivenko and Cantelli strengthened this result by proving uniform convergence of Fn to F by the Glivenko–Cantelli theorem.[2]
A centered and scaled version of the empirical measure is the signed measure
It induces a map on measurable functions f given by
By the central limit theorem, converges in distribution to a normal random variable N(0, P(A)(1 − P(A))) for fixed measurable set A. Similarly, for a fixed function f, converges in distribution to a normal random variable , provided that and exist.
Definition
- is called an empirical process indexed by , a collection of measurable subsets of S.
- is called an empirical process indexed by , a collection of measurable functions from S to .
A significant result in the area of empirical processes is Donsker's theorem. It has led to a study of Donsker classes: sets of functions with the useful property that empirical processes indexed by these classes converge weakly to a certain Gaussian process. While it can be shown that Donsker classes are Glivenko–Cantelli classes, the converse is not true in general.
Example
As an example, consider empirical distribution functions. For real-valued iid random variables X1, X2, ..., Xn they are given by
In this case, empirical processes are indexed by a class It has been shown that is a Donsker class, in particular,
- converges weakly in to a Brownian bridge B(F(x)) .
See also
References
- ^ Mojirsheibani, M. (2007). "Nonparametric curve estimation with missing data: A general empirical process approach". Journal of Statistical Planning and Inference. 137 (9): 2733–2758. doi:10.1016/j.jspi.2006.02.016.
- ^ Wolfowitz, J. (1954). "Generalization of the Theorem of Glivenko-Cantelli". The Annals of Mathematical Statistics. 25: 131–138. doi:10.1214/aoms/1177728852.
Further reading
- Billingsley, P. (1995). Probability and Measure (Third ed.). New York: John Wiley and Sons. ISBN 0471007102.
- Donsker, M. D. (1952). "Justification and Extension of Doob's Heuristic Approach to the Kolmogorov- Smirnov Theorems". The Annals of Mathematical Statistics. 23 (2): 277–281. doi:10.1214/aoms/1177729445.
- Dudley, R. M. (1978). "Central Limit Theorems for Empirical Measures". The Annals of Probability. 6 (6): 899–929. doi:10.1214/aop/1176995384.
- Dudley, R. M. (1999). Uniform Central Limit Theorems. Cambridge Studies in Advanced Mathematics. Vol. 63. Cambridge, UK: Cambridge University Press.
- Kosorok, M. R. (2008). Introduction to Empirical Processes and Semiparametric Inference. Springer Series in Statistics. doi:10.1007/978-0-387-74978-5. ISBN 978-0-387-74977-8.
- Shorack, G. R.; Wellner, J. A. (2009). Empirical Processes with Applications to Statistics. doi:10.1137/1.9780898719017. ISBN 978-0-89871-684-9.
- van der Vaart, Aad W.; Wellner, Jon A. (2000). Weak Convergence and Empirical Processes: With Applications to Statistics (2nd ed.). Springer. ISBN 978-0-387-94640-5.
- Dzhaparidze, K. O.; Nikulin, M. S. (1982). "Probability distributions of the Kolmogorov and omega-square statistics for continuous distributions with shift and scale parameters". Journal of Soviet Mathematics. 20 (3): 2147. doi:10.1007/BF01239992. S2CID 123206522.
External links
- Empirical Processes: Theory and Applications, by David Pollard, a textbook available online.
- Introduction to Empirical Processes and Semiparametric Inference, by Michael Kosorok, another textbook available online.
- v
- t
- e
- Additive process
- Bessel process
- Birth–death process
- Brownian motion
- Cauchy process
- Contact process
- Continuous-time random walk
- Cox process
- Diffusion process
- Dyson Brownian motion
- Empirical process
- Feller process
- Fleming–Viot process
- Gamma process
- Geometric process
- Hawkes process
- Hunt process
- Interacting particle systems
- Itô diffusion
- Itô process
- Jump diffusion
- Jump process
- Lévy process
- Local time
- Markov additive process
- McKean–Vlasov process
- Ornstein–Uhlenbeck process
- Poisson process
- Schramm–Loewner evolution
- Semimartingale
- Sigma-martingale
- Stable process
- Superprocess
- Telegraph process
- Variance gamma process
- Wiener process
- Wiener sausage
- Binomial options pricing model
- Black–Derman–Toy
- Black–Karasinski
- Black–Scholes
- Chan–Karolyi–Longstaff–Sanders (CKLS)
- Chen
- Constant elasticity of variance (CEV)
- Cox–Ingersoll–Ross (CIR)
- Garman–Kohlhagen
- Heath–Jarrow–Morton (HJM)
- Heston
- Ho–Lee
- Hull–White
- Korn-Kreer-Lenssen
- LIBOR market
- Rendleman–Bartter
- SABR volatility
- Vašíček
- Wilkie
- Central limit theorem
- Donsker's theorem
- Doob's martingale convergence theorems
- Ergodic theorem
- Fisher–Tippett–Gnedenko theorem
- Large deviation principle
- Law of large numbers (weak/strong)
- Law of the iterated logarithm
- Maximal ergodic theorem
- Sanov's theorem
- Zero–one laws (Blumenthal, Borel–Cantelli, Engelbert–Schmidt, Hewitt–Savage, Kolmogorov, Lévy)
- Cameron–Martin formula
- Convergence of random variables
- Doléans-Dade exponential
- Doob decomposition theorem
- Doob–Meyer decomposition theorem
- Doob's optional stopping theorem
- Dynkin's formula
- Feynman–Kac formula
- Filtration
- Girsanov theorem
- Infinitesimal generator
- Itô integral
- Itô's lemma
- Karhunen–Loève theorem
- Kolmogorov continuity theorem
- Kolmogorov extension theorem
- Lévy–Prokhorov metric
- Malliavin calculus
- Martingale representation theorem
- Optional stopping theorem
- Prokhorov's theorem
- Quadratic variation
- Reflection principle
- Skorokhod integral
- Skorokhod's representation theorem
- Skorokhod space
- Snell envelope
- Stochastic differential equation
- Stopping time
- Stratonovich integral
- Uniform integrability
- Usual hypotheses
- Wiener space
- Actuarial mathematics
- Control theory
- Econometrics
- Ergodic theory
- Extreme value theory (EVT)
- Large deviations theory
- Mathematical finance
- Mathematical statistics
- Probability theory
- Queueing theory
- Renewal theory
- Ruin theory
- Signal processing
- Statistics
- Stochastic analysis
- Time series analysis
- Machine learning
- List of topics
- Category