The History About Internet Messaging

Published Date: 02 Nov 2017

Because the devices used by sender and receivers of data all not totally reliable, and there is no realistic way to ensure the reliability of data across noisy communication channels, techniques have be developed that not only detect errors, but whenever possible, correct them. The sophistication of these error-correcting schemes have been improved and adapted over time to be implemented in all modern networks of communications.

Suppose a message needs to be sent through a noisy communication channel that is denoted either by Yes (1) or No (0). It can be assumed that there is a constant probability of p (where p<1) that any binary digit transmission can be altered, irrespective and independent or any other messages sent. This is referred to as a binary symmetric channel.

An obvious issue arriving with this system of messages is, when a single bit is received, there is only a 1-p probability that the channel with be reliable at the time. A simple way of increasing reliability, providing p was small, would be to send the message twice:

P(no errors occurring)= (1-p)^2 (Large value)

P(one error occurring)= ((1-p)*p)+((1-p)*p) = 2p*(1-p) (Small value)

P(2 errors occurring)= p*p = p^2 (Very small value)

We would require p<0.5 for the reception of a correct message to be more likely than an error occurring.

If the received message were "11" or "00", it could be deduced that either two or no errors have happened. This we could expect a probability of:

___(1-p)^2___

(1-p)^2 +p^2

This would give a probability if the message was Yes or No.

If the message received was "10" or "01", then an error or bit flip has definitely occurred. If errors occur frequently then having the message resent in this manner is inefficient.

The sender could try sending 3 bits of information, where Yes "111" or No "000", if only one bit flip transpires, it can be deduced with a decent reliability what message was sent. If the message "000" is sent, of some eight possible results, "000", "100", "010", "001" would be decoded accurately as No. That would mean this message had a probability for correct decoding or:

P(no errors occurring)= (1-p)^3 "000"

P(one error occurring)= p*(1-p)^2 "100", "010", "001"

(1-p)^3 + 3*p*(1-p)^2= (1-p)^2 * (1+2p) = 1 - 3p^2 + 2p^3

If p is small, the probability of a decoding error, 3p^2 + 2p^3 is extremely small. This encoding is known as a repetition code. It can quickly improve the reliability over a channel. So much so that if p = 0.001 (1/1000 bits are garbled) this kind of triple repetition would be ensuring that only about 1 out of every 330,000 messages would be incorrectly decoded.

\/////////////////////////////////////////////////////////////////////////////////////////////////////////

Despite the simplicity of these codes, sending entire message three times over is neither practical nor necessarily guarantee error detection. The main aim of coding theory is to find such codes so that reliable error detection and correction is as feasible as possible. There is really no way to totally provide total accuracy in regards to lack of errors within a message, so the aim is to make the probability for accurate decoding as close to 1 as possible.

So far we have only dealt with binary code words, involving a finite sequence of 0s and 1s. If there are q options to make the code from, then in the case of binary codes, we take q=2. It follows that a ternary code word would be a finite sequence of 0s, 1s & 2s, so q=3.

Codewords themselves can be treated as a vectors in the space Fnq= Fq × Fq× . . . Fq

Definition: Let q ∈ N. A q-ary code word is a finite sequence of symbols that form the set or alphabet Fq = {λ1, λ2, . . . , λq}.

In almost all cases this set is thought of as Zq = {0, 1, 2, ..., q −1}

Definition: A set of M code words, where M ∈ N is known as the size of the code.

In this respect, an example would be that all the words of the English language are codes contained within the set {A, B, C, …, Z}

Definition: The Hamming Distance d(x,y), is the number of places that x & y differ, where x, y ∈ Fq

The Hamming Distance satisfies the properties:

d(x,y)=0 ⇐⇒ x=y,

d(x,y) = d(y,x) for all x,y ∈ Fq

d(x,y) ≤ d(x,z) + d(z,y) for all x,y,z ∈ Fq

The third property above is referred to as the triangle inequality, which essentially states the minimum distance required to move from x to y cannot be greater than minimum distance to move from x to y via z.

Letting C be a code in Fnq, then we can define the minimum distance, d(C) = min{ d(x,y) : x, y ∈C, x≠y}. It is beneficial that the distance between the code words are as great as possible.

Definition The number of nonzero integers in q-ary code word x, is referred to as the weight w(x)

If x, y ∈ Zn2, (binary codewords) then d(x, y) = w(x - y), where w( x – y) = w(x) +w(y) -2w(x*y)

Definition A code of length n, containing M code words and having a minimum distance d, has notation (n, M, d).

For example, binary codes of length 4, with 3 code words that have a minimum distance of 2 would be denoted (4,3,2):

C = 0 0 0 0

0 1 0 1

1 0 1 0

With a code such as this, it is possible to detect 1 error, as the code words have a minimum distance greater than 1. But the detection and correction of an error is not possible, since received message with a single error may not closer to one code or the other. (1,0,0,0) could be either (1,0,1,0) or (0,0,0,0) given 1 bit flip. If a larger code is used such as (6,3,3):

C= 0 0 0 0 0 0

1 1 0 0 1 0

1 0 1 1 1 0

Then it is possible to detect up to 2 errors, given that the minimum distance between the code words is greater than 2. It is even possible to detect and correct up to 1 error, as if a bit flip occurs, the resultant message would be closer to the transmitted message than any other. (0 0 0 0 1 0) is closest to (0 0 0 0 0 0) than any other. However if (0 0 0 0 1 0) is the result of more than 1 error occurring, then it is not possible to correct.

Theorem In a symmetric channel, where the probability of an error p>0, there exist code C where:

it is possible to detect t errors in every code word ⇐⇒ d(C) ≥ t + 1

it is possible to correct t errors in any code word ⇐⇒ d(C) ≥ 2t + 1

Proof (1):

Let x be a code word that is received with t or fewer errors, creating a new message y ∈ Fnq.

⇒ d(x,y) = w(x – y) ≤ t < t + 1 ≤ d(C)

The transmitted message, y, cannot be another code word, hence t errors can be detected.

Let C be a code that can detect up to t errors.

d(C)≤ t +1 ⇒ ∃ x,y : d(x,y) ≤ t

Because code word x becomes y after t errors, it is reasonable to deduce that C cannot detect t

errors. This contradicts the premise above, hence d(C) ≥ t + 1.

Proof (2):

Let x be a code word that is received with t or fewer errors, creating a new message y ∈ Fnq, that

satisfies d(x,y)≤ t.

z is another code word such that x≠z,

d(x,z)≥2t + 1 & d(x,z)≤d(x,y)+d(y,z) [Triangle Inequality]

⇒d(y,z)≥d(x,z)-d(x,y)≥2t+1-t=t+1>t≥d(x,y)

So as the message y resembles x more than z, it is possible to identify x correctly.

Let C be a code that can detect up to t errors

d(c) <2t+1⇒∃ x,z : d(x,z) ≤2 t

If d(x,z)≤t, let y=z, so 0=d(y,z)<d(y,x).

If t<d(x,z)≤2t, a new code word w must be construct by changing t components of x that are differing with z to equal z, so 0<d(y,z)≤d(y,x)=t

In both cases it is possible to transmit the code word x, and receive the message of y due to t or fewer errors in communication. However, due to d(y,z)≤d(y,x), y cannot be decoded in a definite way. This contradicts the premise above, hence d(C)≥ 2t+1.

Corollary If the minimum distance of C is d, then C can be used in such a way as to detect up to d-1 errors or correct (d-1)/2 to the nearest integer, in any given code word.

The (n,M,d) code that an individual or institution is striving for has to be efficient in terms of speed, size and reliability. On these conditions, and from what is known already, this means n is required to be as small as possible for swift transmission, M to be large to maximise the total amount of data for being sent, and that d is large to facilitate as much error correction as possible.

Definition: Let Aq(n,d) be the greatest value of M such that a q-ary (n,m,d) code exists.

From the previous codes (4,3,2) and (6,3,3), we know therefore that A2(4, 2)≥3 and A2(6, 3)≥3 respectively.

Theorem For any value of q and n,

Aq(n,1) =q^n

Aq(n,n) =q

Proof (1):

With the minimum distance d=1, it is only required that the code words are distinct from each other. The greatest possible code with this property is the whole of Fnq, which has M=q^n code words.

Proof (2):

With the minimum distance d=n, the code words must be differing from each other in n positions. Specifically, the components in the initial positions must differ, so there can be q code words or fewer. A q-ary repetition codes, like those discussed earlier, of length n is an example of (n,q,) code, so Aq(n,n) =q is possible.

Since d(x, y) = w(x−y)≤n for the distinct code words x,y ∈ Fnq, Aq(n,d) is not defined if d>n. So for d(C) to be defined, there must be no fewer than 2 code words.

Lemma (Reduction Lemma) if some q-ary (n,M,d) code exist with d≥2, then some (n-1, M, d-1) code exists too.

Proof: Let x,y be code words where d(x,y)=d. Identify any column where x and y are distinct from one another. Remove this column from all the code words. As d≥2, the new code words are still distinct and create a (n-1, M, d-1) code.

Theorem (where d is an even value) Let d be even. Then the binary (n,M,d) code exists iff (n-1,M,d-1) code also exists.

The proof follows from the reduction Lemma previous.

Let C be some binary (n-1,M,d-1) code. Let C’ a code of length n that is found when extending each code word x of C by adding a parity bit w(x) mod2. This make the new weight w(x’) of every code word x’ of C’ even.

It follows that d(x,y)=w(x) +w(y) -2w(xy) is even for all pairs of code words x,y in C, therefore d(C’) is even.

d-1=d(C)≤d(C’)≤d. But as d-1 is odd, d(C’) must equal d. hence C’ is an (n,M,d) code.

Corollary (max code size for d that is even)

Given that d is even, A2(n,d)=A2(n−1,d−1)

This means that only when d is odd, do we have to calculate A2(n,d). There is little value in considering codes when d is even when trying to perform error correction.