Log out Go to App
Go to App

Learning Materials

Features

Discover

Chapter 9: Q. 9.5 (page 413)

Let X be a random variable that takes on 5 possible values with respective probabilities .35, .2, .2, .2, and .05. Also, let Y be a random variable that takes on 5 possible values with respective probabilities .05, .35, .1, .15, and .35. (a) Show that H(X) > H(Y). (b) Using the result of Problem 9.13, give an intuitive explanation for the preceding inequality.

Short Answer

Expert verified

The results are

(a)H(X)=2.14,H(y)=2.02

(b)Xis closer to unform distribution as compare toY.

Step by step solution

01

Part (a) Step 1: Given Information

We have to proveH(X)>H(Y).

02

Part (a) Step 2: Simplify

Consider

H(X)=-∑ipilogpi=-0.35log20.35-3·0.2log20.2-0.5=2.14

and on the other side, we have

localid="1651485575012" H(Y)=-∑ipilogpi=-0-05log20.05-0.35log20.35-0.1log20.1-0.15log20.15-0.35log20.35=2.02

So, we have proved H(X)>H(Y).

03

Part (b) Step 1: Given Information

We have to find an intuitive explanation for the preceding inequality.

04

Part (b) Step 2: Explanation

Considering random variable Xhas much more uncertainty than Y, i.e. its distribution is much closer to the uniform distribution as compare toYand from the problem 9.13.we know that uniform distribution has maximal entropy.

Unlock Step-by-Step Solutions & Ace Your Exams!

  • Full Textbook Solutions

    Get detailed explanations and key concepts

  • Unlimited Al creation

    Al flashcards, explanations, exams and more...

  • Ads-free access

    To over 500 millions flashcards

  • Money-back guarantee

    We refund you if you fail your exam.

Start your free trial

Over 30 million students worldwide already upgrade their learning with Vaia!

One App. One Place for Learning.

All the tools & learning materials you need for study success - in one app.

Get started for free

Most popular questions from this chapter

Cars cross a certain point in the highway in accordance with a Poisson process with rate λ = 3 per minute. If Al runs blindly across the highway, what is the probability that he will be uninjured if the amount of time that it takes him to cross the road is s seconds? (Assume that if he is on the highway when a car passes by, then he will be injured.) Do this exercise for s = 2, 5, 10, 20.

Suppose that in Problem 9.2, Al is agile enough to escape from a single car, but if he encounters two or more cars while attempting to cross the road, then he is injured. What is the probability that he will be unhurt if it takes him s seconds to cross? Do this exercise for s = 5, 10, 20, 30.

On any given day, Buffy is either cheerful (c), so-so (s), or gloomy (g). If she is cheerful today, then she will be c, s, or g tomorrow with respective probabilities .7, .2, and .1. If she is so-so today, then she will be c, s, or g tomorrow with respective probabilities .4, .3, and .3. If she is gloomy today, then Buffy will be c, s, or g tomorrow with probabilities .2, .4, and .4. What proportion of time is Buffy cheerful?

Suppose that whether it rains tomorrow depends on past weather conditions only through the past 2 days. Specifically, suppose that if it has rained yesterday and today, then it will rain tomorrow with probability .8; if it rained yesterday but not today, then it will rain tomorrow with probability .3; if it rained today but not yesterday, then it will rain tomorrow with probability .4; and if it has not rained either yesterday or today, then it will rain tomorrow with probability .2. What proportion of days does it rain?

In transmitting a bit from location A to location B, if we let X denote the value of the bit sent at location A and Y denote the value received at location B, then H(X) − HY(X) is called the rate of transmission of information from A to B. The maximal rate of transmission, as a function of P{X = 1} = 1 − P{X = 0}, is called the channel capacity. Show that for a binary symmetric channel with P{Y = 1|X = 1} = P{Y = 0|X = 0} = p, the channel capacity is attained by the rate of transmission of information when P{X = 1} = 1 2 and its value is 1 + p log p + (1 − p)log(1 − p).
See all solutions

Recommended explanations on Math Textbooks

Discrete Mathematics

Read Explanation

Statistics

Read Explanation

Geometry

Read Explanation

Theoretical and Mathematical Physics

Read Explanation

Logic and Functions

Read Explanation

Applied Mathematics

Read Explanation
View all explanations

What do you think about this solution?

We value your feedback to improve our textbook solutions.

Study anywhere. Anytime. Across all devices.

Sign-up for free