b) Briefly explain about i) Non-singular codes ii) Uniquely decodable codes iii)
Instantaneous codes with an example. (3) c) Asource emits 6 messages whose
code words are generated from the set {O, 1, 2}. Can an instantaneous code
with code word lengths for each of the

messages as (|, |, 2, 4, 4, 5] be generated? Justify. (2)
Part B
4. a) State and prove the properties of typical sets. (4)
b) Discuss the consequence of AEP. (3)
c) Briefly explain i) typical set ii) high probability sets. (2)
5. a) Prove that feedback cannot increase the capacity of achannel. (5)

b) ۲۱۱٢ the capacity of asymmetric channel defined by a source emitting two
equally likely messages where the channel transition probabilities for a row in
the channel transition matrix is {0.3, 0.2, 0.5}. (4)

6. a) State and prove the Asymptotic Equipartition theorem. (3)
b) Define Jointly Typical Sequences. State and prove the properties of joint

AEP. (6)
Part C

7. a) Briefly explain Shannon limit. A binary telephone channel which is band
limited from 250 Hz to 3550 Hz has a SNR of 30 GB. Calculate the capacity
of the channel in bps. (4)

b) Evaluate the differential entropy for the exponential density, f(x} = A.
where x20.(4) c) If h(X) represents the differential entropy for the function
f(x), prove that

h(aX) h(X) + logl al. (4)
8. a) State and prove the converse to the rate distortion theorem. (7)
b) Evaluate the rate distortion function for a binary source. (5)
9. a) Derive the differential entropy of a Gaussian distribution. (4)

b) Consider "k" independent Gaussian channels in parallel with a common power
constraint. Explain the process of power distribution among the channels so as
to maximize the capacity. (8)