P.T.O.

Part 3
4. a) Briefly explain i) typical set ii) high probability sets (2)
b) State the Asymptotic Equipartition theorem and hence prove it. (3)
c) State and prove any two properties of typical sets. (2)
d) Discuss the consequence of AEP. (2)
5. 9) State and prove Shannon's channel coding theorem. (6)
b) Derive the capacity of binary symmetric channel. (3)
6. a) State and prove the source-channel theorem. (6)

b) Find the capacity and the maximising probability distribution of the channel whose

transition matrix is given below: (The input and outputs axe binary)

(3)
0.5 05

Part C

7. a) Evaluate the differential entropy for the exponential density, f(x) = k.e-)-x 20. (3)
b) Derive the differential entropy of a normal distribution. (3)

c) Derive the chain rule for differential entropy. (3)

d) Prove that h(aX) = h(X) + log lal where h(X) represents the differential entropy. (3)

8. a) State and prove the converse to the rate distortion theorem. (7)

b) Evaluate the rate distortion function for a binary source. (5)

9. a) Briefly explain Shannon limit. A telephone channel which is bandlimited from 300 Hz to
3600 Hz, has a SNR = 33 GB, calculate the capacity of the telephone channel in bits per
second. (4)
b) Explain the water filling in Parallel Gaussian channels. (8)