Well-behaved statistic

Although the term well-behaved statistic often seems to be used in the scientific literature in somewhat the same way as is well-behaved in mathematics (that is, to mean "non-pathological") it can also be assigned precise mathematical meaning, and in more than one way. In the former case, the meaning of this term will vary from context to context. In the latter case, the mathematical conditions can be used to derive classes of combinations of distributions with statistics which are "well-behaved" in each sense.
First Definition: The variance of a well-behaved statistical estimator is finite and one condition on its mean is that it is differentiable in the parameter being estimated.
Second Definition: The statistic is monotonic, well-defined, and locally sufficient.
Conditions for a Well-Behaved Statistic: First Definition.
More formally the conditions can be expressed in this way. formula_1 is a statistic for formula_2 that is a function of the sample, formula_3. For formula_1 to be "well-behaved" we require:
formula_5: Condition 1
formula_6 differentiable in formula_7, and the derivative satisfies:
formula_8: Condition 2
Conditions for a Well-Behaved Statistic: Second Definition.
In order to derive the distribution law of the parameter "T", compatible with formula_9, the statistic must obey some technical properties. Namely, a statistic "s" is said to be well-behaved if it satisfies the following three statements:
The remainder of the present article is mainly concerned with the context of data mining procedures applied to statistical inference and, in particular, to the group of computationally intensive procedure that have been called algorithmic inference.
Algorithmic inference.
In algorithmic inference, the property of a statistic that is of most relevance is the pivoting step which allows to transference of probability-considerations from the sample distribution to the distribution of the parameters representing the population distribution in such a way that the conclusion of this statistical inference step is compatible with the sample actually observed.
By default, capital letters (such as "U", "X") will denote random variables and small letters ("u", "x") their corresponding realizations and with gothic letters (such as formula_23) the domain where the variable takes specifications. Facing a sample formula_24, given a sampling mechanism formula_25, with formula_26 scalar, for the random variable "X", we have
The sampling mechanism formula_28, of the statistic "s", as a function ? of formula_16 with specifications in formula_14 , has an explaining function defined by the master equation:
for suitable seeds formula_32 and parameter ?
Example.
For instance, for both the Bernoulli distribution with parameter "p" and the exponential distribution with parameter ? the statistic formula_33 is well-behaved. The satisfaction of the above three properties is straightforward when looking at both explaining functions: formula_34 if formula_35, 0 otherwise in the case of the Bernoulli random variable, and formula_36 for the Exponential random variable, giving rise to statistics
and
"Vice versa", in the case of "X" following a continuous uniform distribution on formula_39 the same statistics do not meet the second requirement. For instance, the observed sample formula_40 gives
formula_41. But the explaining function of this "X" is formula_42.
Hence a master equation formula_43 would produce with
a "U" sample formula_44 and a solution formula_45. This conflicts with the observed sample since the first observed value should result greater than the right extreme of the "X" range. The statistic formula_46 is well-behaved in this case.
Analogously, for a random variable "X" following the Pareto distribution with parameters "K" and "A" (see Pareto example for more detail of this case),
and
can be used as joint statistics for these parameters.
As a general statement that holds under weak conditions, sufficient statistics are well-behaved with respect to the related parameters. The table below gives sufficient / Well-behaved statistics for the parameters of some of the most commonly used probability distributions.