Log out Go to App
Go to App

Learning Materials

Features

Discover

Chapter 9: Q.9.17 (page 413)

Show that for any discrete random variable Xand functionf

H(f(X))≤H(X)

Short Answer

Expert verified

The given statement is proved below.

Step by step solution

01

Given Information

We have to prove that

H(f(X))≤H(X)

02

simplify

Consider two random variables, XandfX. Using the formula about the entropy of two random variables, we have that

HX,fX=HX+HXfX

Observe that knowing X, random variable fXbecomes absolutely deterministic, i.e. there is no more uncertainty about its value. Hence HXfX=0which implies that HX,fX=HX. On the other hand, we can also write

HX,fX=HfX+HfXX≥HfX

which finally implies that

HfX≤HX

Unlock Step-by-Step Solutions & Ace Your Exams!

  • Full Textbook Solutions

    Get detailed explanations and key concepts

  • Unlimited Al creation

    Al flashcards, explanations, exams and more...

  • Ads-free access

    To over 500 millions flashcards

  • Money-back guarantee

    We refund you if you fail your exam.

Start your free trial

Over 30 million students worldwide already upgrade their learning with Vaia!

One App. One Place for Learning.

All the tools & learning materials you need for study success - in one app.

Get started for free

Most popular questions from this chapter

Cars cross a certain point in the highway in accordance with a Poisson process with rate λ = 3 per minute. If Al runs blindly across the highway, what is the probability that he will be uninjured if the amount of time that it takes him to cross the road is s seconds? (Assume that if he is on the highway when a car passes by, then he will be injured.) Do this exercise for s = 2, 5, 10, 20.

Determine the entropy of the sum that is obtained when a pair of fair dice is rolled.

Suppose that in Problem 9.2, Al is agile enough to escape from a single car, but if he encounters two or more cars while attempting to cross the road, then he is injured. What is the probability that he will be unhurt if it takes him s seconds to cross? Do this exercise for s = 5, 10, 20, 30.

Prove that if X can take on any of n possible values with respective probabilities P1, ... ,Pn, then H(X) is maximized when Pi = 1/n, i = 1, ... , n. What is H(X) equal to in this case?

A pair of fair dice is rolled. Let

X=1ifthesumofthediceis60otherwise

and let Y equal the value of the first die. Compute (a) H(Y), (b) HY(X), and (c) H(X, Y).
See all solutions

Recommended explanations on Math Textbooks

Applied Mathematics

Read Explanation

Logic and Functions

Read Explanation

Probability and Statistics

Read Explanation

Geometry

Read Explanation

Theoretical and Mathematical Physics

Read Explanation

Mechanics Maths

Read Explanation
View all explanations

What do you think about this solution?

We value your feedback to improve our textbook solutions.

Study anywhere. Anytime. Across all devices.

Sign-up for free