Word-valued sources: An ergodic theorem, an AEP, and the conservation of entropy

A word-valued source Y = Y₁,Y₂,̇ ̇ ̇ is discrete random process that is formed by sequentially encoding the symbols of a random process ${\bf X} = X₁,X₂ ̇ ̇ ̇ with codewords from a codebook. These processes appear frequently in information theory (in particular, in the analysis of source-coding algorithms), so it is of interest to give conditions on X and for which Y will satisfy an ergodic theorem and possess an asymptotic equipartition property (AEP). In this paper, we prove the following: 1) if X is asymptotically mean stationary (AMS), then Y will satisfy a pointwise ergodic theorem and possess an AEP; and 2) if the codebook is prefix-free, then the entropy rate of Y is equal to the entropy rate of X normalized by the average codeword length.