UNIT-5

ELEMENTS OF CODING THEORY

• Coding theory deals essentially with the transmission of a message through a medium
such that whatever errors the medium may introduce, could be detected and if possible
corrected, in an efficient manner, at the receiver's end.

• The message to be sent is encoded into a code word using a encoding rule.

• Then the coded version is transmitted over a channel.

• When the coded word passes through the channel, errors may occur due to weather
interference, electrical problems and so on and it becomes distorted.

• The decoder corrects the error in the distorted message and decode the corrected
version using decoding rule into the original message.
Definitions:
Message: The basic unit of information, called a message, is a
finite sequence of characters from a finite alphabet.
Word: A sequence of symbols from an alphabet is often referred
to as word.
Code: A code is a collection of words that are to be used to
represent distinct messages.
Code word: A word in a code is called a codeword.
Block Code: A block code is a code consisting of words that are of
the same length.
 In a binary alphabet {0, 1}, there are 2n binary words of length n.
For n≥1, let

• A(n, m) code for a binary message is a tuple (Bm, Bn, E, D),

where Bm is the set of all binary m-tuples, Bn is the set of
all binary n-tuples (n > m) and

E: Bm →Bn and D: R → Bm where R⊆Bn.

• The function E and D are called encoding function and the

decoding function of a (n, m) code.

• If b∈ Bm then E(b) is called the codeword representing b.

• A nonempty subset of Bm is called a set of message words.
• Let X⊆ Bm be a set of message words, then E(X) is called a
set of code words.
• Then code words are transmitted through the transmission
channel.
• Let X' be the set of received words after transmission.
• These received words are decoded by the decoding function
D, then D(X') is the set of decoded words.
• Suppose that words to be transmitted through a
communication channel are all the members of the set of
binary words of length 3

• i.e, B3={000, 001, 010, 011, 100, 101, 110,111}. If the

word 100 is transmitted and an error occurs in the second
digit so that the received word is 110.

• There is no way of detecting the error because 110 is a

member of the set of words which one might expect to
receive.

• This shows that if we are to detect errors, an incorrectly

transmitted word must not be a member of the set of
words.
• In general, codes are classified into two types

• 1. Error-detecting code whose purpose is simply to detect the

presence of errors, and

• 2. Error correcting code in which an attempt is made to

correct errors which occurs because of noise in the
transmission channel.

• One first maps the sequence of characters to be transmitted

message bit into a longer sequence by adding extra bit of the
same characters by an encoding scheme.

• By inspecting the additional information in the extra

characters, the receiver is then able to detect and/or correct
errors which have introduced by noise during transmission.
 For example, let us consider the (4, 3) parity-check code.
Then B3 is the set of message words. Then

are code words.

• If the transmission channel transmits E(010) = 0100, then there is
an odd number of 1's and we know that an odd number of errors
(at least one) has occurred.
• This code is an error detecting code. However the error cannot be
corrected because it is not possible to tell which digit is at fault.
• Suppose a=101 and transmission makes an error in the sixth digit and the received
word is b=E(101)=101100101 but this is not a code word. Thus we can detect a single
error and can also correct the error. This code can also detect two errors, but can correct
only one error.

• Here we decode b, by examining the first, fourth and seventh positions to see which
signal appears more times. Hence 1 occurs all three times so the first entry in the
decoded message is 1. Next we examine second, fifth and eighth positions and here 0
occurs all three times so the second entry in the decoded message is 0.

• Continuing with entries in the third, sixth and ninth position, 1 occurs twice and 0
occurs once, hence the result for third entry in the decoded message is 1. So, the
transmitted message was 101.
Generally, designs for communication systems are analyzed
mathematically using simplified models for the channel. Let
binary, represented by 0's and 1's be transmitted sequentially
through a channel to a receiver as shown in Fig. 19.1.
This simple model for the channel is called the binary symmetric
channel.
To maximize the transmission rate we want enough code words
to encode a given message alphabet but at the same time the words
should be no longer than necessary to achieve sufficient error
detection and/or correction.

The good error control coding scheme should have the following
features:
(i) Capability to correct more number of errors.
(ii) Fast and efficient scheme for encoding the message
(iii) Fast and efficient scheme for decoding the received message
(iv) Maximum transfer of information bits per unit time.
Hamming Weight and Distance
• Let A denote the set of all binary sequence of length n. If x be a
word in A, then Hamming weight w(x) is defined by- the
number of 1's in x and denoted by w(x).

• For example, w(0101) = 2 and w(1011) = 3. For x and y in A,

the Hamming distance d(x,y) between x and y is defined by the
number of positions where the code words have the different
symbols.

• For example, d(1011, 1111)=1 and d(1110000, 0001111)=7.

Thus the distance between x=x1x2 ... xn and y=y1y2 ... yn is the
number of values of i such that xi≠yi that is, the number of
positions in which x and y differ.
Properties of Distance Function
• Let x, y and z be elements of Bm. Then
1. d(x, y) = d(y, x)
2. d(x, y)≥0
3. d(x, y) = 0 if and only if x = y .
4. d(x + a, y + a)= d(x, y) i.e. the distance between two words
is unaltered if we add the same word to both of them.
5. d(x, y)≤ d(x, z) + d(z, y)

• The minimum distance of an encoding function E : Bm→Bn is

the minimum of the distances between all distinct pairs of code
words, that is,
min {d(E(x), E(y)) : x, y ∈Bm}
We can represent the code symbols using vectors in a linear
vector space. For n = 3, the representation is shown in Fig.l9.2
• Thus, we compute d(xi,y) and conclude xk was the transmitted
word if d(xk , y) is the smallest among all distances computed.

• This means if an erroneous code word is received, we should

assume that the nearest code word was sent in the sense that it
differs in the fewer digits.

• This is known as the minimum-distance decoding criterion

or nearest - neighbor decoding principle.
 Group (Linear) Codes
How group structure plays a role in coding theory ?

• It is easy to show that (Bm, +),and (Bn, +)are commutative groups under
component wise addition (mod 2) of words.
• Note that every element in Bm and Bn has its own inverse and zero word is
the identity element.
• Let E : Bm →Bn be an encoding function .
• The code C = E(Bm) is called a group code if C is a subgroup of Bn.
• From the definition of the subgroup , C the subset of Bn consisting of all code
words is a subgroup of Bn if
1) The identity element in Bn is in C.
2) If x, y ∈ C, then x + y ∈ C.
3) If x in C, then its inverse is also in C.
In this case (iii) is obvious since every element in Bn is its own inverse.
Decoding Methods

(a) Maximum-likelyhood Decoding

(b) Minimum distance decoding criteria
The Parity-Check and Generator Matrix
• Encoding and decoding function by matrices.

• Earlier, all the code words are written.

• We, Create codewords by addition of other codewords, which means that there is no
need to hold every code word.

• If there are m bits of information, all we need to hold m linearly independent

codewords i.e. a set of codewords none of which can be produced by linear
combinations of two or more codewords in the set.

• The easiest way to find m linearly independent codewords is to choose those which
have 1 in just one of the m positions and 0 in other m-1of the first m positions.
• For (7, 4) encoding function, we need just the four codewords.

1000011
0100101
0010110
0001111
• If we wish to obtain the codeword for 0111, we add together the second, third and
fourth codewords to the list to give 0111100.
• The process of encoding by addition can be represented in matrix form by
v = uG
where u is the message , v the codeword and G the generator matrix.
• So we can represent the code by the generator matrix as given below:

• If we wish to encode the message 0111, then

v = [0111]G = [0111100]

• The coding of the message can thus be considered as a multiplication of 1x m vector (the
message) and a m x n generator matrix G which can be written in normal form as
G = [Im | A]

• where Im represents m x m identity matrix and A is a m x (n - m) matrix of parity check

bits .
• A code where the generator matrix has this normal form is called a systematic code and
has the property that the first m symbols of a code word are identical to the message.
• The remaining n - m symbols are the parity checks.
Since
Example: