en.wikipedia.org/wiki/Maximum_entropy_probability_distribution
2 corrections found
{ − ∞ , ∞ }
This support entry is incorrect for the logistic distribution. Its support is all real numbers, usually written ℝ or (−∞, ∞), not the two-point set {−∞, ∞}.
Full reasoning
In the table, this expression appears in the Support column for the logistic distribution. That is mathematically incorrect.
The logistic distribution is a continuous distribution defined for every real value. Standard references describe it as being supported over the set of real numbers / for (y \in \mathbb{R}). By contrast, the notation ({-\infty, \infty}) denotes a set containing only two elements, negative infinity and positive infinity, not the whole real line.
So the correct support should be written as (\mathbb{R}) or ((-\infty, \infty)), not ({-\infty, \infty}).
2 sources
- LogisticDistribution — Wolfram Documentation
LogisticDistribution [ μ , β ] represents a continuous statistical distribution defined and supported over the set of real numbers...
- 18.8 Logistic distribution | Stan Functions Reference
If μ ∈ ℝ and σ ∈ ℝ+ , then for y ∈ ℝ , Logistic(y|μ,σ) = ...
Taking limits α → 1 and α → 0 , respectively, yields H ( q ) ≥ H ( p ) , H ( p ′ ) .
This inference is false. A mixture q = αp + (1−α)p′ need not have entropy at least as large as both H(p) and H(p′).
Full reasoning
The preceding inequality,
[
H(q) \ge \alpha H(p) + (1-\alpha)H(p')
]
is the standard concavity property of entropy for the mixture (q = \alpha p + (1-\alpha)p'). But the sentence quoted goes further and concludes that for the same fixed mixture (q) one gets
[
H(q) \ge H(p),; H(p').
]
That conclusion does not follow, and it is false in general.
A concrete counterexample uses the discrete entropy formula (H(X)=-\sum_i p_i \log p_i):
- Let (p=(1/2,1/2)), so (H(p)=\ln 2 \approx 0.693).
- Let (p'=(9/10,1/10)).
- Take (\alpha=1/2). Then
[
q = \tfrac12 p + \tfrac12 p' = (0.7,0.3).
] - Its entropy is
[
H(q)=-(0.7\ln 0.7 + 0.3\ln 0.3) \approx 0.611.
]
So in this example,
[
H(q) \approx 0.611 < 0.693 \approx H(p),
]
which directly contradicts the claim that (H(q) \ge H(p)) and (H(q) \ge H(p')).
What is true is only the concavity inequality above; it does not imply that a nontrivial mixture has entropy at least as large as each component individually.
2 sources
- 8. Basic Elements of Information Theory - Lecture Notes on Fundamentals of Data Analysis
In the case of a discrete variable: H(X) = - Σ_x P(x) log P(x).
- Maximum entropy probability distribution - Wikipedia
From basic facts about entropy, it holds that H(q) ≥ α H(p) + (1−α) H(p′). Taking limits α → 1 and α → 0, respectively, yields H(q) ≥ H(p), H(p′).