Log out Go to App
Go to App

Learning Materials

Features

Discover

Chapter 9: Q. 9.13 (page 413)

Prove that if X can take on any of n possible values with respective probabilities P1, ... ,Pn, then H(X) is maximized when Pi = 1/n, i = 1, ... , n. What is H(X) equal to in this case?

Short Answer

Expert verified

The required statement is proved in below step.

Step by step solution

01

Given Information

We have to find H(X)and prove the given statement.

02

Simplify

Considering the problem of maximization

H(p1,...,pn)=-∑i=1npilog2pi

with condition ∑i=1npi=1.

Considering the Langrangian function

L(p1,...pn,λ)=-∑i=1npilog2pi-λ∑Pii=1n-1

With differentiation, we will get the system of equations

localid="1648209529913" ∂ℒ∂pi=-log2Pi+1log2-λ=0,i=1,...,n∂ℒ∂λ=∑pii=1n-1=0

If we look at the first equality a little bit better, we have that

log2p1+1log2=-λ=log2pj+1log2

for every i,j.it implies that pi=pjfor every iand j. Hence, putting these information in the last equality, we have

np1-1=0⇒p1=1n

which implies pi=1n.

Hence, we have proved the statement.

Unlock Step-by-Step Solutions & Ace Your Exams!

  • Full Textbook Solutions

    Get detailed explanations and key concepts

  • Unlimited Al creation

    Al flashcards, explanations, exams and more...

  • Ads-free access

    To over 500 millions flashcards

  • Money-back guarantee

    We refund you if you fail your exam.

Start your free trial

Over 30 million students worldwide already upgrade their learning with Vaia!

One App. One Place for Learning.

All the tools & learning materials you need for study success - in one app.

Get started for free

Most popular questions from this chapter

asdsadasdsadsad

A pair of fair dice is rolled. Let

X=1ifthesumofthediceis60otherwise

and let Y equal the value of the first die. Compute (a) H(Y), (b) HY(X), and (c) H(X, Y).

Determine the entropy of the sum that is obtained when a pair of fair dice is rolled.

A certain person goes for a run each morning. When he leaves his house for his run, he is equally likely to go out either the front or the back door, and similarly, when he returns, he is equally likely to go to either the front or the back door. The runner owns 5 pairs of running shoes, which he takes off after the run at whichever door he happens to be. If there are no shoes at the door from which he leaves to go running, he runs barefooted. We are interested in determining the proportion of time that he runs barefooted. (a) Set this problem up as a Markov chain. Give the states and the transition probabilities. (b) Determine the proportion of days that he runs barefooted.

This problem refers to Example 2f.

(a) Verify that the proposed value of πj satisfies the necessary equations.

(b) For any given molecule, what do you think is the (limiting) probability that it is in urn 1?

(c) Do you think that the events that molecule j, j Ú 1, is in urn 1 at a very large time would be (in the limit) independent?

(d) Explain why the limiting probabilities are as given.
See all solutions

Recommended explanations on Math Textbooks

Probability and Statistics

Read Explanation

Decision Maths

Read Explanation

Statistics

Read Explanation

Logic and Functions

Read Explanation

Theoretical and Mathematical Physics

Read Explanation

Geometry

Read Explanation
View all explanations

What do you think about this solution?

We value your feedback to improve our textbook solutions.

Study anywhere. Anytime. Across all devices.

Sign-up for free