CS492

CS492
CS492 - Data Security and Cryptography
Part B
MACS, USP.
Hugh Anderson
March 3, 2000
Preface
The semester II, 1999 course “CS492 - Data Security and Cryptography” at Mathematics and
Computing Science, University of the South Pacific, is taught by Dr. Russel Pears (Course
supervisor) and Hugh Anderson.
The course is concerned with techniques for storing data safely, for transporting data across
(possibly) hostile networks, and securing computer systems.
This encompasses both simple encoding methods for ensuring transport of digital data across an
analog network, right through to cryptographic1 methods for ensuring secrecy and authenticity.
In part B of the course, Hugh will concentrate on implementation details of these systems. Most
topics in part B will have a practical component, involving either hands-on, or demonstrations.
These notes along with supporting materials are found at: http://opo.usp.ac.fj/˜hugh/Public/cs492/.
1
The Free-On-Line-Dictionary-Of-Computing is found at http://wombat.doc.ic.ac.uk/foldoc/index.html - it defines cryptography as:
The practise and study of encryption and decryption - encoding data so that it can only be decoded
by specific individuals. A system for encrypting and decrypting data is a cryptosystem.
i
ii
Contents
1
Information and communication
1
1.1
Analog and digital . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1
1.2
Fourier analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2
1.2.1
Fourier series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
1.2.2
Fourier transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
1.2.3
Convolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
Modulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
1.3.1
Case study: amplitude modulation . . . . . . . . . . . . . . . . . . . . .
8
1.3.2
Baseband digital encoding . . . . . . . . . . . . . . . . . . . . . . . . .
9
Information theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
1.3
1.4
1.4.1
Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.4.2
Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.4.3
Shannon and Nyquist . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.5
Huffman encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.6
Case study: modems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.6.1
2
MNP5 and V.42bis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Error detection
2.1
Cyclic redundancy check codes . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.1.1
2.2
19
Hardware representation . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Case study: ethernet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
iii
CONTENTS
iv
3
Error correction
3.1
Code types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.2
BER and noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.2.1
Hamming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.4
Reed-Solomon codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.5
Convolutional codes
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.5.1
Convolutional encoding . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.5.2
Viterbi decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Case study: ECC encoders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Encryption and authentication
4.1
4.2
5
A very bad ECC transmission scheme: repetition . . . . . . . . . . . . . 25
3.3
3.6
4
23
Symmetric key systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.1.1
Ciphertext . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.1.2
Product Ciphers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.1.3
DES - Data Encryption Standard
4.1.4
Case study: Amoeba capabilities . . . . . . . . . . . . . . . . . . . . . . 36
5.2
. . . . . . . . . . . . . . . . . . . . . 35
Public key systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.2.1
Diffie-Hellman key agreement . . . . . . . . . . . . . . . . . . . . . . . 36
4.2.2
Encryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.2.3
Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.2.4
RSA (Rivest, Shamir, Adelman) . . . . . . . . . . . . . . . . . . . . . . 39
4.2.5
RSA coding algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.2.6
Case study: PGP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
System (in)security
5.1
33
41
Ethical concerns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
5.1.1
Why study this? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
5.1.2
Ethics and computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
5.1.3
Professional codes of ethics . . . . . . . . . . . . . . . . . . . . . . . . 43
Insecurity - threats and protection . . . . . . . . . . . . . . . . . . . . . . . . . 45
5.2.1
Non-cryptographic cracking . . . . . . . . . . . . . . . . . . . . . . . . 45
CONTENTS
5.2.2
5.3
CERTő Incident Note IN-99-04 . . . . . . . . . . . . . . . . . . . . . . 47
5.4
NSA - National Security Agency . . . . . . . . . . . . . . . . . . . . . . . . . . 49
5.5
C2 security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Insecurity case studies
6.1
6.2
6.3
PkZip stream cipher fix . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
UNIX base security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
6.2.1
Crypt algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
6.2.2
Brute force cracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
6.2.3
Dictionary cracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
6.2.4
UNIX base security fix . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
Microsoft base security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
6.3.1
LanManager encryption . . . . . . . . . . . . . . . . . . . . . . . . . . 58
6.3.2
NT encryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
6.3.3
Challenge-response protocol . . . . . . . . . . . . . . . . . . . . . . . . 58
6.3.4
Attack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
6.3.5
Microsoft base security fix . . . . . . . . . . . . . . . . . . . . . . . . . 60
Security systems
7.1
7.2
7.3
61
Kerberos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
7.1.1
Kerberos protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
7.1.2
Weaknesses
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
ssh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
7.2.1
RSA key management . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
7.2.2
Port forwarding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
7.2.3
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
SSL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
7.3.1
7.4
53
PkZip stream cipher . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
6.1.1
7
Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
CERT - Computer Emergency Response Team . . . . . . . . . . . . . . . . . . . 47
5.3.1
6
v
UN-SSL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
PGPfone
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Chapter 1
Information and communication
When studying the transfer and storage of data, there are some underlying physical laws, representations and constraints to consider.
• Is the data analog or digital?
• What limits are placed on it?
• How is it to be transmitted?
• How can you be sure that it is correct?
1.1 Analog and digital
An analog signal is a continuous valued signal. A digital signal is considered to only exist at
discrete levels.
5
sin(x)+4
(sin(x)>=0)+1
real(int(sin(x)*5))/10
4
3
2
1
0
-1
-10
-8
-6
-4
-2
0
1
2
4
6
8
10
CHAPTER 1. INFORMATION AND COMMUNICATION
2
5
sin(x)+4
(sin(3*x)/3)+2
sin(x)+(sin(3*x)/3)
4
1.0
3
2
0.333
0.2
1
0
-1
-10
f
-8
-6
-4
-2
0
2
(a) Time domain
4
6
8
3f
5f
10
(b) Frequency domain
Figure 1.1: Sum of sine waveforms.
The (time domain) diagrams are commonly used when considering signals. If you use an oscilloscope, the display normally shows something like that shown on the previous page. The plot
is amplitude versus time. With any analog signal, the repetition rate (if it repeats) is called the
frequency, and is measured in Hertz (pronounced hurts, and written Hz). The peak to peak signal
level is called the amplitude.
The simplest analog signal is called the sine wave. If we mix these simple waveforms together,
we may create any desired periodic waveform. In figure 1.1, we see the sum of two sine waves
- one at a frequency of 1,000Hz, and the other at three times the frequency (3,000Hz). The
amplitudes of the two signals are 1 and 13 respectively, and the sum of the two waveforms
shown, approximates a square wave. If we were to continue summing these waves, in the same
progression, the resultant waveform would be a square wave
∞
X
1
sin(2πnf ) (for odd n) ⇒ a square wave of frequency f
n
n=1
We may also represent these signals by frequency domain diagrams, which plot the amplitude
against frequency. This alternative representation is also shown in figure 1.1.
1.2 Fourier analysis
One way of representing any simple periodic function is as a sum of simple sine (and cosine)
waveforms. This representation method is known as Fourier Analysis after Jean-Baptiste Fourier,
who first showed the technique.
1.2. FOURIER ANALYSIS
3
The Fourier method can be viewed as a transformation between equivalent time domain and
frequency domain representations. A piecewise continuously differentiable periodic function in
the time domain may be transformed to a discrete aperiodic function in the frequency domain.
If our time domain function is f (t) then we normally write the corresponding frequency domain
function as F (ω), and we use the symbol ↔ to represent the transformation:
f (t) ↔ F (ω)
There are various flavours of Fourier analysis depending on the types of functions in each domain. The table below summarizes the methods used.
Time domain
Continuous, periodic
Continuous, aperiodic
Discrete, periodic
Discrete, aperiodic
Frequency domain
↔
↔
↔
↔
Description
Discrete, aperiodic
Continuous, aperiodic
Discrete, periodic
Continuous, periodic
Fourier series
Fourier transform
Discrete Fourier series
Discrete Fourier transform
1.2.1 Fourier series
We start with the equation for constructing an arbitrary periodic waveform g(t)1 :
g(t) = a0 +
∞
X
n=1
an cos(2πnf t) +
∞
X
bn sin(2πnf t)
n=1
f is the fundamental frequency of the waveform, and an and bn are the amplitudes of the sine and
cosine components at each of the harmonics of the fundamental. Since an and bn are the only
unknowns here, it is easy to see that if we know the fundamental frequency, and the amplitudes
an and bn , we may reconstruct the original signal g(t).
For any g(t) we may calculate a0 , an and bn by first noting that the integral over the interval
[0, T ] will be zero for the summed terms:
Z T
1
1
g(t) dt where T =
a0 =
T 0
f
and by multiplying both sides of the equation by sin(2πkf t), and noting that:
š
Z T
0 f or k 6= n
sin(2πkf t) sin(2πnf t) dt =
T
f or k = n
0
2
1
Note that the Fourier series may also be given in complex exponential form as:
∞
1 X
cn ejωn t
g(t) =
T n=−∞
CHAPTER 1. INFORMATION AND COMMUNICATION
4
11
sin(x)+10
sin(x)+(sin(3*x)/3)+8
sin(x)+(sin(3*x)/3)+(sin(5*x)/5)+6
sin(x)+(sin(3*x)/3)+(sin(5*x)/5)+(sin(7*x)/7)+4
10
9
8
7
6
5
4
3
-10
-8
-6
-4
-2
0
2
4
6
8
10
Figure 1.2: Successive approximations to a square wave.
and
Z
T
sin(2πkf t) cos(2πnf t) dt = 0
0
We can then integrate to get
2
bk =
T
Z
T
g(t) sin(2πkf t) dt
0
Similarly, by multiplying by cos(2πkf t), we get
2
ak =
T
Z
T
g(t) cos(2πkf t) dt
0
Example: A bipolar square wave gives ak = 0, and
b1 = 1 b2 = 0 b3 = 13
k=1 k=2 k=3
b4 = 0 b5 = 51
k=4 k=5
b6 = 0 ...
k = 6 ...
We re-create our waveform by summing the terms:
1
1
1
4
(sin(2πf t) + sin(6πf t) + sin(10πf t) + sin(14πf t) + ...)
π
3
5
7
In figure 1.2, we see four plots, showing the resultant waveforms if we sum the first few terms in
the series. As we add more terms, the plot more closely approximates a square wave.
Note that there is a direct relationship between the bandwidth of a channel passing this signal,
and how accurate it is. If the original (square) signal had a frequency of 1,000Hz, and we were
1.2. FOURIER ANALYSIS
5
attempting to transmit it over a channel which only passed frequencies from 0 to 1,000Hz, we
would get a sine wave.
Another way of stating this is to point out that the higher frequency components are important
- they are needed to re-create the original signal faithfully. If we had two 1,000Hz signals, one
a triangle, one a square wave - if they were both passed through the 1,000Hz bandwidth limited
channel above, they would look identical (a sine wave).
1.2.2 Fourier transform
With aperiodic waveforms, we consider the Fourier Transform of our function f (t), which is
the function F (ω) given by
Z
∞
F (ω) =
This transform may be inverted to give
1
f (t) =
2π
f (t)e−jωt dt
−∞
Z
∞
F (ω)ejωt dω
−∞
In figure 1.3 we see various simple transforms. Note that if a function in one domain is widened,
its transform narrows.
1.2.3 Convolution
One of the important theorems in Fourier analysis is the convolution theorem, which states that:
If f (t) and g(t) are two functions with Fourier transforms F (ω) and G(ω), then
the Fourier transform of the convolution f (t) ? g(t) is the product of the Fourier
transforms of the functions F (ω) and G(ω), and vice versa.
f (t) ? g(t) ↔ F (ω) × G(ω)
f (t) × g(t) ↔ F (ω) ? G(ω
The convolution k(t) of f (t) and g(t) may be expressed as
Z T
1 +2
f (t − τ )g(τ ) dτ
k(t) = f (t) ? g(t) = (f ? g)(t) =
T − T2
but it also has a graphical interpretation. We can use convolution to easily predict the functions
that result from complex signal filtering or sampling2 .
In figure 1.4, we see a sine wave and a sampling window, each with their own Fourier transform.
By multiplying the two waveforms, we end up with a single cycle of the sine wave, and we can
deduce its frequency domain representation by convolving the two Fourier transforms.
2
In class, we will use this technique to demonstrate the impossibility of a perfect filter.
CHAPTER 1. INFORMATION AND COMMUNICATION
6
f(t)
f(t)
t
F( ω)
t
F( ω)
ω
ω
(b) Higher frequency sine wave
(a) Low frequency sine wave
f(t)
f(t)
t
F( ω)
t
F( ω)
ω
ω
(d) Pulse
(c) Square wave
f(t)
f(t)
t
F( ω)
t
F( ω)
ω
(e) Shorter pulse
ω
(f) The sinc function
Figure 1.3: Sample plots showing functions and their transforms.
1.3. MODULATION
7
F( ω)
f(t)
ω
t
G( ω)
g(t)
ω
t
F( ω) * G(ω )
f(t).g(t)
t
ω
Figure 1.4: Window sampling.
1.3 Modulation
A baseband signal is one in which the data component is directly converted to a signal and
transmitted. When the signal is imposed on another signal, the process is called modulation.
We may modulate for several reasons:
• The media may not support the baseband signal
• We may wish to use a single transmission medium to transport many signals
We use a range of modulation methods, often in combination:
• Frequency modulation - frequency shift keying (FSK)
• Amplitude modulation
• Phase modulation - phase shift keying (PSK)
• Combinations of the above (QAM)
CHAPTER 1. INFORMATION AND COMMUNICATION
8
12
sin(x*20)+8
sin(abs(x)+x)+4
(1+0.8*sin(abs(x)+x))*sin(x*20)
10
8
6
4
2
0
-2
-10
-8
-6
-4
-2
0
2
4
6
8
10
Figure 1.5: Amplitude modulation
1.3.1 Case study: amplitude modulation
The process of amplitude modulation involves varying the amplitude of a high frequency carrier
signal with a lower frequency modulation signal. In figure 1.5, we see three plots - at the top a
carrier signal, with a frequency 10 times that of the modulation signal, shown in the middle plot.
The lowest plot shows the product of the two signals - an amplitude modulated signal.
An initial look at this might suggest that the frequency domain plot of this signal would have
components at the carrier and modulation frequencies. However - lets do the maths - if our
carrier is fc (t) = cos ωc t and the modulation signal is fm (t) = cos ωm t, then our multiplicative
100% amplitude modulated signal is:
g(t) = (1 + cos ωm t) cos ωc t
= cos ωc t + cos ωm t cos ωc t
1
1
= cos ωc t + cos(ωc + ωm )t + cos(ωc − ωm )t
2
2
Our final frequency domain plot looks like that shown in figure 1.6. In general, any modulating
signal spectrum is shifted to become the upper and lower sidebands of the carrier signal, as
predicted by the convolution theorem.
Incidentally, only one of the sidebands is needed, and the other may be removed or filtered out
without losing any information. Doing this reduces the signal energy by 25%. In addition, the
carrier itself may be removed, reducing the signal energy by a further 50%.
The cumulative effects of these two optimizations is commonly known as SSB (Single side band
with suppressed carrier), and uses 21 the bandwidth of an equivalent amplitude modulated channel
with 41 the power.
1.4. INFORMATION THEORY
9
Amplitude
Amplitude
Frequency
Frequency
Fm
Fc-Fm
Fc
Fc+Fm
Fm
Fc-Fm
Fc
Fc+Fm
Figure 1.6: Frequency domain plots of amplitude modulated signals.
1.3.2 Baseband digital encoding
The simplest encoding scheme is just to use a low level for a zero bit, and a high level for a one
bit. As long as both ends of a channel are synchronized in some manner, we can transfer data. On
the other hand, if the ends of the channel are not synchronized we might use a simple encoding
scheme, such as Bipolar or Manchester encoding, to transfer synchronizing (clock) information
on the same channel.
TIME
TIME
BITS
BITS
CODE
CODE
CLOCK
CLOCK
RECVD
RECVD
Bipolar
Manchester
In Bipolar encoding, a 1 is transmitted with a positive pulse, a 0 with a negative pulse. Since
each bit contains an initial transition away from zero volts, a simple circuit can extract this clock
signal. This is sometimes called return to zero encoding.
In Manchester (phase) encoding, there is a transition in the center of each bit cell. A binary 0
causes a high to low transition, a binary 1 is a low to high transition. The clock retrieval circuitry
is slightly more complex than before.
1.4 Information theory
The term information is commonly understood. Consider the following two sentences:
1. The sun will rise tomorrow.
2. The Fiji rugby team will demolish the All Blacks the next time they play.
Question: Which sentence contains the most information3 ?
3
Most of us would have no hesitation in stating that the second statement contains more information than the
first statement. In general, the less predictable the message, the more information in it.
CHAPTER 1. INFORMATION AND COMMUNICATION
10
Information
Source
Transmitter
Receiver
Destination
(or sink)
Channel
Noise
Source
Figure 1.7: Model of communication system.
Two early researchers - Nyquist (1924) and Hartley (1928) laid the foundation for a formal
treatment of information.
Hartley showed that the information content of a message is proportional to the logarithm of the
number of possible messages. He used morse encodings, but the same can be applied to binary
encodings - if we wish to encode integers between 1 and n, we need log2 n bits.
Shannon developed a more complete mathematical treatment of communication and information
in a seminal paper [1]. The paper develops a general theory of communication over a noisy
channel. We model the communication system in schematic form in figure 1.7.
In the following presentation, we assume that the unit of information is the binary digit (or bit), as
most computer systems use this representation. There is a strong parallel found in other scientific
areas - for example the study of statistical mechanics has a similar concept of entropy.
1.4.1 Entropy
In our communication model, the units of transmission are called messages, constructed from an
alphabet of (say) n symbols x ∈ {x1 , . . . , xn } each with a probability of transmission Px . We
associate with each symbol x a quantity Hx which is a measure of the information associated
with that symbol.
1
Hx = Px log2
Px
If the probability of occurence of each symbol is the same, by adding these for all symbols x we
can derive Hartley’s result, that the average amount of information transmitted in a single symbol
(the source entropy) is
H(X) = log2 n
where X is a label referring to each of the source symbols x1 , . . . , xn . However, if the probability
of occurence of each symbol is not the same, we derive the following result, that the source
entropy is
n
X
1
Pxi log2
H(X) =
Px i
i=1
1.4. INFORMATION THEORY
11
Shannon’s paper shows that H determines the channel capacity required to transmit the desired
information with the most efficient coding scheme. Our units for entropy can be bits/second
or bits/symbol, and we also sometimes use unit-less relative entropy measures (relative to the
entropy of the system if all symbols were equally likely). We can also define the entropy of a
continuous (rather than the discrete) distribution over x with density p(x) as
Z ∞
1
p(x) log2
H(x) =
dx
p(x)
−∞
Example: If we had a source emitting two symbols, 0 and 1, with equal probabilities of occuring, then the entropy of the source is
H(X) =
n
X
Pxi log2
i=1
1
Pxi
= 0.5 log2 2 + 0.5 log2 2
= 1
Example: If we had a source emitting two symbols, 0 and 1, with probabilities of 1 and 0, then
the entropy of the source is
H(X) =
n
X
Pxi log2
i=1
1
Pxi
= log2 1 + 0 ∗ log2 0
= 0
Note that:
lim y log2
y→0
1
=0
y
The information rate for a source providing r symbols/sec is R = rH(X) bits/sec, and the
channel capacity is
log2 N (T )
C = lim
T →∞
T
Example: If we were transmitting a sequence of letters A,B,C,D,E and F with probabilities
1
1 1 1 1 1
, , , , and 16
, the entropy for the system is
2 4 16 16 16
1
1
4
log2 2 + log2 4 +
log2 16
2
4
16
= 0.5 + 0.5 + 1.0
= 2 bits/symbol
H(X) =
Lets compare two encodings, first a fixed size 3-bit code, and then a more complex code:
12
CHAPTER 1. INFORMATION AND COMMUNICATION
Symbol
3-bit code
Complex code
A
000
0
B
001
10
C
010
1100
D
011
1101
E
100
1110
F
101
1111
The average length of the binary digits needed to encode a typical sequence of symbols using the 3-bit code is
H(X) =
n
X
i=1
Pxi • sizeof(xi )
1
4
1
∗3+ ∗3+
∗3
2
4
16
= 1.5 + 0.75 + 0.75
= 3 bits/symbol
=
But we can do much better if we encode using the other encoding. The average
length of the binary digits needed to encode a typical sequence of symbols using the
complex encoding is
H(X)
=
n
X
i=1
=
=
=
Pxi • sizeof(xi )
1
1
4
∗1+ ∗2+
∗4
2
4
16
0.5 + 0.5 + 1.0
2 bits/symbol
Example: If our source was transmitting 0 and 1 bits with equal probability, but the received
data was corrupted 50% of the time, we might reason that our rate r(X) of information
transmission was 0.5, because half of our data is getting through correctly.
However, a better argument is to consider the difference between the entropy of the source
and the conditional entropy of the received data:
r(X) = H(X) − Hy (X)
where Hy (X) is the conditional entropy of the received data.
Hy (X) =
=
and H(X) =
so r(X) =
=
0.5 ∗ log2 2 + 0.5 ∗ log2 2
1
1
(shown before)
H(X) − Hy (X)
0 bits/symbol
This is a much better measure of the amount of information transmitted when you consider that you could model the system just as effectively by using a random bit generator
connected to the receiver.
1.4. INFORMATION THEORY
13
1.4.2 Redundancy
The ratio of the entropy of a source H(X) to what it would be if the symbols had equal probabilities H 0 (X), is called the relative entropy. We use the notation Hr (X), and
Hr (X) =
H(X)
H 0 (X)
The redundancy of the source is 1 − Hr (X).
R(X) = 1 − Hr (X)
If we look at English text a symbol at a time4 , the redundancy is about 0.5. This indicates that it
should be simple to compress English text by about 50%.
1.4.3 Shannon and Nyquist
In white noise, the distribution of the power densities for a signal with noise power N is a
gaussian function:
1
1
2
e− 2N x
p(x) =
(2πN )
If p(x1 , ..., xn ) is a gaussian density distribution function for n samples x1 , . . . , xn (i.e. sampled
white noise), then Shannon derives the power entropy H(X) of the function:
H(X) = W log2 2πeN
the maximum possible entropy for a given average power N .
Assume we have a composite signal with entropy H(Y ), consisting of an information source
H(S) and a noise source with entropy H(N ). If the noise is independent of the signal, our
channel capacity is
C = H(Y ) − H(N )
If all these sources are essentially random (i.e. they have maximum entropy), then
H(Y ) = W log2 2πe(S + N )
H(N ) = W log2 2πeN, and so
S
C = W log2 (1 + )
N
This result is commonly used for noisy (thermal noise) channels, expressed in the following way:
Maximum BPS = W log2 (1 +
4
That is, without considering letter sequences.
S
)
N
bits/sec
14
CHAPTER 1. INFORMATION AND COMMUNICATION
Example: If we had a telephone system with a bandwidth of 3,000 Hz, and a S/N of 30db (about
1024:1)
D = 3000 ∗ log2 1025
≈ 3000 ∗ 10
≈ 30000 bps
This is a typical maximum bit rate achievable over the telephone network.
Nyquist shows us that the maximum data rate over a limited bandwidth (W) channel with V
discrete levels is:
Maximum data rate = 2W log2 V bits/sec
For example, two-Level data cannot be transmitted over the telephone network faster than 6,000
BPS, because the bandwidth of the telephone channel is only about 3,000Hz.
Example: If we had a telephone system with a bandwidth of 3,000 Hz, and using 256 levels:
D = 2 ∗ 3000 ∗ log2 256
= 6000 ∗ 8
= 48000 bps
In these equations, the assumption is that the relative entropies of the signal and noise are a
maximum (that they are random). In practical systems, signals rarely have maximum entropy,
and we can do better - there may be methods to compress the data5 .
1.5 Huffman encoding
An immediate question of interest is “What is the minimum length bit string that may be used to
compress a string of symbols?”.
The Huffman encoding minimizes the bit length given the frequency of occurence of each symbol6 . The resultant bit string in the best case will be the length predicted from the calculation of
the source entropy.
A huffman encoder uses a binary tree with symbols arranged at the leafs such that each leaf has
a unique prefix. In the example in figure 1.8, the letter E is encoded by following the path from
1.5. HUFFMAN ENCODING
15
0
0
1
1
0
E
1
T
0
A
0
1
O
N
1
S
Figure 1.8: Tree encoding for Huffman codes.
the tree root “00”. This is the shortest path and shortest encoding, since E is the most commonly
used letter in English text.
We can see that less common characters such as A, O, N and S, use longer bit strings. Our
algorithm for encoding is simple - we calculate the tree encoding knowing the frequency of each
letter, and just construct a table for each symbol:
Symbol
Coding
E
T
A
O
N
S
00
10
010
011
110
111
To decode a Huffman encoded string, we traverse the tree as each bit is received, taking a left
path or a right path according to the bit being a 0 or a 1. When we reach the leaf, we have our
symbol.
5
Note: we must also differentiate between lossy and lossless compression schemes. A signal with an entropy
of 0.5 may not be compressed more than 2:1 unless you use a lossy compression scheme. JPEG and Wavelet
compression schemes can achieve huge data size reductions without visible impairment of images, but the restored
images are not the same as the original ones - they just look the same. The lossless compression schemes used in
PkZip, gzip or GIF files (LZW) cannot achieve compression ratios as high as that found in JPEG.
6
Note that it presupposes knowledge about these frequencies.
CHAPTER 1. INFORMATION AND COMMUNICATION
16
1.6 Case study: modems
When we transmit data over a media which does not support one of these simple encoding
schemes, we may have to modulate a carrier signal, which can be carried over the media we
are using. The telephone network supports only a limited range of frequencies.
We use a range of modulation methods, often in combination:
• AM Amplitude modulation
• FM/FSK Frequency modulation - frequency shift keying
• PM/PSK Phase modulation - phase shift keying
With a bandwidth of only 3,000Hz Nyquist shows us that there is no point in sampling more
than 6,000 samples per second and if we only sent 1 bit per change in signal, we could only
send 6,000 bits/sec. Common modulation methods focus on sending multiple bits per change
to increase data rates (up to the maximum determined by noise - Shannon). The most common
method is phase modulation, shown below:
10
0
90
180
270
00
01
00
00
01
10
11
We can also send different amplitudes at the different phases. The following phase plots indicate
useful phase/amplitude values:
1.6. CASE STUDY: MODEMS
17
90
90
0
0
180
180
270
270
These schemes use multiple amplitudes and phases. They are called QAM7 . The one on the left
has 2 amplitudes and 4 phases giving a total of 3 bits per change. In the other example, we are
sending 4 bits/change (4 bits/baud).
Common modem encoding standards are:
• V.32 bis (14k4 => 14.4k) - 6 bits/baud => 64 points in the constellation.
• V.34 bis (28k8 => 28.8k) - 7 bits/baud => 128 point in the constellation.
Most modems do some or all of the following to reduce errors and improve speed.
• Add a parity bit to each 8 bits.
• Carefully choose where to place bit patterns in the constellation to reduce errors.
• use (software) compression of the data (MNP5 and V.42bis)
1.6.1 MNP5 and V.42bis
MNP5 and V42.bis are compression schemes commonly used on modems. MNP5 suffers from
the unfortunate property that it will expand data with maximum or near-maximum entropy (instead of compression). V42.bis does not have this property - it uses a large dictionary, and will
not try to compress an already compressed stream.
MNP5 uses two different compression methods, switching between them as appropriate. The
methods are:
• Adaptive frequency encoding
• Run-length encoding
7
Quadrature Amplitude Modulation.
CHAPTER 1. INFORMATION AND COMMUNICATION
18
Run length encoding sends the bytes with a byte count value, and doubles the size of a data stream
with maximum entropy. Adaptive frequency encoding uses a similar scheme as that shown in
our complex-code on page 11:
3-bit header
Body size
Total code size
Number of codewords
000
1
bit
4 bits
2
001
1
bit
4 bits
2
010
2 bits
5 bits
4
011
3 bits
6 bits
8
100
4 bits
7 bits
16
101
5 bits
8 bits
32
110
6 bits
9 bits
64
111
7 bits
10 bits
128
We can see from this that 34 of our codewords are larger than they would be if we did not use this
encoding scheme, and with an input stream with an even spread of data (i.e. maximum entropy),
our encoding will increase the size of data.
Chapter 2
Error detection
It is possible to use ad-hoc methods to generate check sums over data, but it is probably best to
use standard systems, with guaranteed and well understood properties, such as the CRC1 .
2.1 Cyclic redundancy check codes
The CRC is commonly used to detect errors. One way of considering CRC systems is to treat
the stream of transmitted bits as a representation of a polynomial with coefficients of 1:
10110 = x4 + x2 + x1 = F (x)
Checksum bits are added to ensure that the final composite stream of bits is divisible by some
other polynomial g(x). We can transform any stream F (x) into a stream T (x) which is divisible
by g(x). If there are errors in T (x), they take the form of a difference bit string E(x) and the
final received bits are T (x) + E(x).
When the receiver gets a correct stream, it divides it by g(x) and gets no remainder. The question
is: How likely is that T (x) + E(x) will also divide with no remainder?
Single bits? - No a single bit error means that E(x) will have only one term (x1285
say). If the generator polynomial has xn + ... + 1 it will never divide evenly.
Multiple bits? - Various generator polynomials are used with different properties.
Must have one factor of the polynomial being x1 + 1, because this ensures all odd
numbers of bit errors (1,3,5,7...).
Some common generators:
1
Cyclic Redundancy Code.
19
CHAPTER 2. ERROR DETECTION
20
• CRC-12 - x12 + x11 + x3 + x2 + x1 + 1
• CRC-16 - x16 + x15 + x2 + 1
• CRC-32 - x32 + x26 + x23 + x22 + x16 + x12 + x11 + x10 + x8 + x7 + x5 + x4 + x2 + 1
• CRC-CCITT - x16 + x12 + x5 + 1
This seems a complicated way of doing something, but polynomial long division is easy when
all the coefficients are 1. Assume we have a generator g(x) of x5 + x2 + 1 (100101) and the
stream F (x): 101101011.
Our final bit stream will be 101101011xxxxx. We divide F (x) by g(x), and the remainder is
appended to F (x) to give us T (x):
1010.01000
100101 )101101011.00000
100101
100001
100101
1001.00
1001.01
1000
We append our remainder to the original string, giving T (x) = 10110101101000.
When this stream is received, it is divided but now will have no remainder if the stream is received
without errors.
2.1.1 Hardware representation
In the previous section we mentioned that polynomial long division is easy when all the coefficients are 1. This is because a simple electronic circuit can perform the calculation continuously
on a stream of bits.
The circuit is constructed from exclusive-or gates (XOR gates), and shift registers.
C
A
B
D
A XOR B
S/R
Q
The XOR gate output is the exclusive-or function of the two input values. The shift register
output Q changes to the input value when there is a rising clock signal.
2.2. CASE STUDY: ETHERNET
21
D
C
Q
A
B
A XOR B
0
1
0
1
↑
↑
↓
↓
0
1
D
D
0
0
1
1
0
1
0
1
0
1
1
0
Table 2.1: Logic functions for XOR and the shift register.
Simple circuits may be constructed from these two gates which can perform polynomial long
division. In the circuit shown in the figure below, there are five shift registers, corresponding to
a check sequence length of 5 bits, and a polynomial generator of length 6. In this example, the
generator polynomial is 1001012 .
Clock
C
Data
D
S/R
C
Q
D
S/R
C
Q
D
XOR
S/R
C
Q
D
S/R
C
Q
D
S/R
Q
XOR
If the hardware system has “all 0s”, and we input the stream 101101011, we get the following
states:
Input data
D0
D1
D2
D3
D4
...
1
1
0
1
0
1
...
0
1
1
0
1
0
0
0
0
1
1
0
1
1
0
0
0
1
1
0
1
0
0
0
0
1
1
0
0
0
0
0
0
1
1
Note
Initial state
First bit
Second bit
Third bit
(And so on...)
2.2 Case study: ethernet
Ethernet is the term for the protocol described by ISO standard 8802.3. It is in common use for
networking computers, principally because of its speed and low cost.
The maximum size of an ethernet frame is 1514 bytes3 , and a 32-bit FCS is calculated over the
2
The left-most shift register corresponds to the least significant bit of the generator polynomial.
1500 bytes of data, a source and destination address each of six bytes, and a two byte type identifier. The frame
also has a synchronizing header and trailer which is not checked by a CRC.
3
CHAPTER 2. ERROR DETECTION
22
full length of the frame.
The FCS used is:
• CRC-32 - x32 + x26 + x23 + x22 + x16 + x12 + x11 + x10 + x8 + x7 + x5 + x4 + x2 + 1
On a 10Mbps ethernet, a full length frame is transferred in less than 1 mS, and the polynomial
long division using the above generator polynomial is done efficiently using a 32 stage shift
register found in the ethernet circuitry. This circuitry calculates the FCS as each bit is received,
and is used both for
• constructing a FCS when transmitting, and
• checking the FCS when receiving.
Chapter 3
Error correction
There are various methods used to correct errors. An obvious and simple one is to just detect
the error and then do nothing, assuming that something else will fix it. This method is fine when
something else is able to fix the error, but is of no use if there is no something else!
• In data communication protocols, it is common to just ignore errors that are received,
while acknowledging correct data. If an error is received, the lack of an acknowledgement
eventually leads to a retransmission after some timeout period. This technique is called
ARQ (for Automatic Repeat reQuest).
• With computer memory, we have a large number of extremely small gates storing bits of
information. Radiation (gamma rays, X rays) can cause the gates to change state from time
to time, and modern computer memory corrects these errors.
When we do this second sort of correction, it is called FEC (Forward Error Control) to differentiate it from ARQ systems.
3.1 Code types
We can divide error correcting codes (ECC) into continuous and block-based types. Convolutional encodings are used for continuous systems, and the common block-based codes are:
• Hamming codes (for correcting single bit errors),
• Golay codes (for correcting up to three bit errors), and
• Bose-Chaudhuri-Hocquenghem (BCH ) codes (for correcting block errors).
Different types of error correcting codes can be combined to produce composite codes. For
example, Reed-Solomon block-codes are often combined with convolutional codes to improve
all-round performance. In this combined setup, the convolutional code corrects randomly distributed bit errors but not bursts of errors while the Reed-Solomon code corrects the burst errors.
23
CHAPTER 3. ERROR CORRECTION
24
System
Wiring of internal circuits
Memory chips
Hard disk
Optical fibre
Co-axial cable
Optical disk (CD)
Telephone system
Error rate (errors/bit)
10−15
10−14
10−9
10−9
10−6
10−5
10−4
Table 3.1: Rates of errors for various systems.
3.2 BER and noise
When designing a system, we may have to achieve a specified bit-error-rate (BER). This BER
generally depends on the type of data. For example, video data may require a very low BER
(10−7 ) whereas speech may be acceptable with a BER of 10−4 . In figure 3.1, we see the raw error
rates for various data storage and communication systems.
In communication systems, BER depends on the signal-to-noise ratio (SNR), as we saw in chapter 1. We can determine the theoretical channel capacity knowing the SNR1 using our equations
from section 1.4.1.
• If the BER is 0.01, the channel capacity C ' 0.92 bits/symbol.
• If the BER is 0.001, the channel capacity C ' 0.99 bits/symbol.
• If the BER is 0, the channel capacity C = 1 bits/symbol.
The theoretical maximum channel capacity is quite close to the perfect channel capacity, even if
the BER is high. We have a range of ways of reducing BER on a particular bandwidth channel.
We can increase the signal (power), or reduce the noise (often not possible), or use ECC.
The benefit of error correcting codes is that they can improve the received BER without increasing the transmitted power. This performance improvement is measured as a system gain.
Example: Consider a system without ECC giving a BER of 0.001 with a S/N ratio of 30dB
(1000:1). If we were to use an ECC codec, we might get the same BER of 0.001 with a
S/N ratio of 20dB (100:1). We say that the system gain due to ECC is 10dB (10:1).
1
If the signal to noise is 1000:1, then our probability of bit error is 0.001.
3.3. HAMMING
25
3.2.1 A very bad ECC transmission scheme: repetition
An initial scheme to correct transmission errors might be to just repeat bits2 .
Data:
0 1 0 0 1 1 1 1 ...
Transmit: 000111000000111111111111...
If we send three identical bits for every bit we wish to transmit, we can then use a voting system
to determine the most likely bit. If our natural BER due to noise was 0.01, with three bits we
would achieve a synthetic BER of 0.0001, but our channel capacity is reduced to about C =
0.31 bits/symbol.
We can see from this that the rate of transmission using repetition has to approach zero to achieve
more and more reliable transmission. However we know from section 1.4.1 that the theoretical
rate should be equal to or just below the channel capacity C. Convolutional and other encodings
can achieve rates of transmission close to the theoretical maximum.
3.3 Hamming
Hamming codes are block-based error correcting codes. In Part A of this course, some properties
of hamming codes are examined. In this part we derive the inequality used to determine how
many extra hamming bits are needed for an arbitrary bit string.
The hamming distance is a measure of how FAR apart two bit strings are. If we examine two bit
strings, comparing each bit, the hamming distance is just the number of different bits at the same
location in the two bit strings. In the following case, we determine that there are three different
bits, and so the hamming distance is 3.
A:
B:
A XOR B:
0 1 0 1 1 1 0 0 0 1 1 1
0 1 1 1 1 1 1 0 0 1 0 1
0 0 1 0 0 0 1 0 0 0 1 0
If we had two bit strings X and Y representing two characters, and the hamming distance between any two codes was d, we could turn X into Y with d single bit errors.
• If we had an encoding scheme (for say ASCII characters) and the minimum hamming
distance between any two codes was d + 1, we could detect d single bit errors3 .
• We can correct up to d single bit errors in an encoding scheme if the minimum hamming
distance is 2d + 1.
2
Note: there is no point in repeating bits twice. you must repeat three times, or 5 times, and then vote to decide
the best value.
3
Because the code d bits away from a correct code is not in the encoding.
CHAPTER 3. ERROR CORRECTION
26
If we now encode m bits using r extra hamming bits to make a total of n = m + r , we can count
how many correct and incorrect hamming encodings we should have. With m bits we have 2m
unique messages - each with n illegal encodings, and:
(n + 1)2m
(m + r + 1)2m
m+r+1
m+r+1
≤ 2n
≤ 2n
≤ 2n−m
≤ 2r
We solve this inequality, and then choose R, the next integer larger than r.
Example: If we wanted to encode 8 bit values (m = 8) and be able to recognise single bit errors:
8+r+1
9
r
R
≤
2r
≤ 2r − r
' 3.5
=
4
3.4 Reed-Solomon codes
Reed-Solomon codes are block-based error correcting codes which are particularly good at correcting bursts (sequences) of bit errors. They are found in a wide range of digital communications
and storage applications. Reed-Solomon codes are used to correct errors in digital wireless applications such as wireless LAN systems, and low Earth orbit (LEO) satellite communication
systems.
Reed-Solomon codes belong to the BCH family of block codes, in which the encoder processes
a discrete block of data to produce an encoded block (or codeword).
A Reed-Solomon code is specified as
• RS(n,k) with s-bit symbols.
This means that the encoder takes k data symbols of s bits each and adds parity symbols to make
an n symbol There are n − k parity symbols of s bits each.
A Reed-Solomon decoder can correct up to t symbols that contain errors in a codeword, where
2t = n − k
3.5. CONVOLUTIONAL CODES
27
Example: A popular Reed-Solomon code is RS(255,223) with 8-bit symbols. Each codeword
contains 255 code word bytes, of which 223 bytes are data and 32 bytes are parity. In this
example, n = 255, k = 223, and s = 8. When these figures are plugged into the above
equation, we can see that
2t = 32
and so t = 16
The Reed-Solomon decoder in this example can correct any 16 symbol errors in the codeword.
Said in another way, errors in up to 16 bytes anywhere in the codeword can be automatically
corrected. In the worst case, 16 bit errors may occur, each in a separate symbol (byte) so that the
decoder corrects 16 bit errors. In the best case, 16 complete byte errors occur so that the decoder
corrects 16 x 8 bit errors.
Given a symbol size s, the maximum codeword length n for a Reed-Solomon code is n = 2s − 1.
For example, the maximum length of a code with 8-bit symbols is 255 bytes.
The amount of processing power required to encode and decode Reed-Solomon codes is proportional to the number of parity symbols for each codeword. A large value means that a large
number of errors can be corrected but requires more computation than a small value.
3.5 Convolutional codes
Convolutional codes are designed to operate continuously and so are especially useful in data
transmission systems. The convolutional encoder operates on a continuous stream of data using
a shift-register to produce a continuous encoded output stream.
The output bit sequence depends on previous sequences of bits. The resultant received bit sequence can be examined for the most likely correct output sequence, even when modified with
an arbitrary number of errors.
This encoding technique is computationally inexpensive, and is commonly used in radio modems.
Convolutional codes are effective for correcting some types of bit errors, particularly the type of
error distribution produced by Gaussian noise. However, these codes are not good at correcting
burst errors, which are longer sequences of errors.
3.5.1 Convolutional encoding
The length of shift register used for a convolutional code is known as the constraint length , and
it determines the maximum number of sequential input bits that can affect the output. The code
rate Rcode is the ratio of the input symbol size to output encoding size:
k
Rcode =
n
CHAPTER 3. ERROR CORRECTION
28
C
C
Data In
D
S/R
Q
D
S/R
C
Q
D
S/R
Q
Data Out
Figure 3.1: Sample convolutional encoder.
An example convolutional encoder with Rcode = 21 , and constraint length 3 is shown in figure
3.1. This coder produces two bits for every single bit of input, and the resultant tree of state
changes repeats after three bits - that is, it only has four distinct states, labelled A, B, C and D in
the diagram below4 .
A 00
A 00
B 11
A 00
C 01
B 11
D 10
(000)
A 10
C 01
B 01
B 11
C 11
D 11
D 00
A 00 (000)
B 11 (001)
C 01 (010)
D 10
A 10
B 01
C 11
D 00
A 00
B 11
C 01
D 10
A 10
B 01
C 11
D 00
(011)
(100)
(101)
(110)
(111)
(000)
(001)
(010)
(011)
(100)
(101)
(110)
(111)
We normally show this in a trellis diagram, which more clearly shows the repetition of the four
states:
4
Note: In these diagrams, we take the upper path for an input of 0 and the lower path for an input of 1.
3.5. CONVOLUTIONAL CODES
A
00
11
00
29
00
11
B
11
11
11
10
10
10
10
01
10
01
01
10
11
D
00
11
01
C
00
00
01
10
11
00
01
01
10
11
00
01
01
10
11
00
00
If we were to input the sequence 011010, we would get the following trace through the trellis,
with the bit sequence output as 001110110101:
A
00
11
10
11
01
01
B
C
D
It is easy to see that there are only certain paths through the trellis diagram, and it is possible
to determine the most likely path, even with large numbers of bit errors. A rate 12 convolutional
encoding can often reduce errors by a factor of 102 to 103 .
CHAPTER 3. ERROR CORRECTION
30
Pseudo code for an encoder might be:
Foreach bit bx do
Case state of
A: Case bx of
0: begin
1: begin
end; {case}
B: Case bx of
0: begin
1: begin
end; {case}
C: Case bx of
0: begin
1: begin
end; {case}
D: Case bx of
0: begin
1: begin
end; {case}
end;
endForeach
transmit(’00’); state := A end;
transmit(’11’); state := B end;
transmit(’01’); state := C end;
transmit(’10’); state := D end;
transmit(’10’); state := A end;
transmit(’01’); state := B end;
transmit(’11’); state := C end;
transmit(’00’); state := D end;
3.5.2 Viterbi decoding
The Viterbi algorithm tries to find the most likely received data sequence, by keeping track of the
four most likely paths through the trellis. For each path, a running count of the hamming distance
between the received sequence and the path is maintained.
Once we have received the first three codes, we start only selecting those paths with a lower
hamming distance. For each of the nodes A..D, we look at the hamming distances associated
with each of the paths, and only select the one with the lower hamming value.
If two merging paths have the same hamming distance, we choose the upper one.
We might be able to implement this algorithm with the following code:
//The four current strings of bits, with their hamming path distances.
streamA, streamB, streamC, StreamD : bitstring;
hamA,
hamB,
hamC,
hamD
: integer;
//Do the following for each received bit pair abx
Foreach bitpair abx do
if (hamA+hamming(abx ,’00’)>=(hamC+hamming(abx ,’10’) then begin
streamA=streamA+’0’; hamA:=hamA+hamming(abx ,’00’)
end else begin
streamA=streamC+’0’; hamA:=hamC+hamming(abx ,’10’)
end;
if (hamA+hamming(abx ,’11’)>=(hamC+hamming(abx ,’01’) then begin
streamB=streamA+’1’; hamB:=hamA+hamming(abx ,’11’)
end else begin
streamB=streamC+’1’; hamB:=hamC+hamming(abx ,’01’)
end;
if (hamB+hamming(abx ,’01’)>=(hamD+hamming(abx ,’11’) then begin
streamC=streamB+’0’; hamC:=hamB+hamming(abx ,’01’)
end else begin
streamC=streamD+’0’; hamC:=hamD+hamming(abx ,’11’)
end;
if (hamB+hamming(abx ,’10’)>=(hamD+hamming(abx ,’00’) then begin
streamD=streamB+’1’; hamD:=hamB+hamming(abx ,’10’)
end else begin
streamD=streamD+’1’; hamD:=hamD+hamming(abx ,’00’)
end;
EndForEach
At any stage in this procedure, we can stop the process, and the most likely received string is the
one with the lowest hamming code.
3.6. CASE STUDY: ECC ENCODERS
31
3.6 Case study: ECC encoders
A finite or Galois field is a group of elements with arithmetic operations in which elements
behave differently than usual. The result of adding two elements from the field is another element in the field. Reed-Solomon encoders and decoders need to carry out this sort of arithmetic
operations.
A number of commercial hardware implementations exist for Reed-Solomon encoding and decoding. They may use off-the-shelf ICs (Integrated Circuits) that encode and decode ReedSolomon codes. These ICs tend to support a certain amount of programmability (for example,
RS(255, k) where t = 1 to 16 symbols).
Example: The COic5127A from Co-Optic Inc, contains a modern high data rate programmable
Reed Solomon encoder that will encode blocks of up to 255 eight bit symbols to provide
corrections of up to 10 errors per code block at data rates up to 320 Mbs. The output code
block will contain the unaltered original data symbols followed by the generated parity
symbols.
The chip supports encoding rates from 0 to 320 Mbs, and comes in a 68 Pin J leaded plastic
chip carrier.
Reed-Solomon codecs can also be implemented in software, the major difficulty being that
general-purpose processors do not support Galois field arithmetic operations. For example, to
implement a Galois field multiply in software requires a test for 0, two log table look-ups, modulo
add, and anti-log table look-up.
However, software implementations can operate reasonably quickly, and a modern software
codec can decode:
Code
Rate
RS(255, 251)
RS(255, 239)
RS(255, 223)
12 Mb/s
2.7 Mb/s
1.1 Mb/s
Viterbi decoders are commonly used in conjunction with trellis modulation in most modern high
speed modems.
32
CHAPTER 3. ERROR CORRECTION
Chapter 4
Encryption and authentication
Security and Cryptographic systems act to reduce failure of systems due to the following threats:
Interruption - attacking the availability of a service (Denial of Service).
Interception - attacks confidentiality.
Modification - attacks integrity.
Fabrication - attacks authenticity. Note that you may not need to decode a signal to fabricate it
- you might just record and replay it.
Encoding and ciphering systems have been in use for thousands of years (hence the Caesar
Cipher). Systems developed before 1976 had a common identifying characteristic: If you knew
how to encipher the plaintext, you could always decipher it1 .
I then told her the key-word, which belonged to no language, and I saw her surprise.
She told me that it was impossible, for she believed herself the only possessor of that
word which she kept in her memory and which she had never written down.
I could have told her the truth - that the same calculation which had served me for
deciphering the manuscript had enabled me to learn the word - but on a caprice it
struck me to tell her that a genie had revealed it to me. This false disclosure fettered
Madame d’Urfé to me. That day I became the master of her soul, and I abused my
power.
Casanova (1757), quote.
We call these systems symmetric key systems.
1
And vice-versa of course.
33
CHAPTER 4. ENCRYPTION AND AUTHENTICATION
34
4.1 Symmetric key systems
P
Ki[P]
(Plaintext)
P
X
X
Ki
Ki
(Plaintext)
Symmetric key systems are generally considered insecure, due to the difficulty in distributing
keys.
4.1.1 Ciphertext
These systems encode the input stream using a substitution rule:
Code
Encoding
A
B
C
D
...
Q
V
X
W
...
The S-box (Substitution-Box) encodes n bit numbers to other n bit numbers and can be represented by the permutation. This is an S-box:
2:4
Permutation
4:2
(3,4,2,1)
Ciphertext is easily breakable, particularly if you know the likely frequency of each of the codes.
In the English language, the most common letters are: “E T A O N I S H R D L U...” (from most
to least common), and we may use the frequency of the encrypted data to make good guesses at
the original plaintext.
4.1. SYMMETRIC KEY SYSTEMS
35
4.1.2 Product Ciphers
We have seen two types of cipher: If you use both types at once, you have a product cipher which
is generally harder to decode, especially if the P box has differing numbers of input and output
lines (1 to many, 1 to 1 or many to 1).
4.1.3 DES - Data Encryption Standard
DES was first proposed by IBM using 128 bit keys, but its security was reduced by NSA (the
National Security Agency) to a 56 bit key (presumably so they could decode it in a reasonable
length of time). At 1ms/GUESS. It would take 1080 years to solve 128 bit key encryption. The
DES Standard gives a business level of safety, and is a product cipher.
The (shared) 56 bit key is used to generate 16 subkeys, which each control a sequenced P-box or
S-box stage. DES works on 64 bit messages.
Note: If you intercept the key, you can decode the message. However, there are about 1017 keys.
There are several modes of operation in which DES can operate, some of them better than others.
The US government specifically recommends not using the weakest simplest mode for messages,
the Electronic Codebook (ECB) mode. They recommend the stronger and more complex Cipher
Feedback (CFB) or Cipher Block Chaining (CBC) modes.
Despite this, most of the commercial encryption packages use ECB mode.
DES is available as a library on both UNIX and Microsoft-based systems. There is typically a
des.h file, which must be included in any C source using the DES library:
#include “des.h”
//
// - Your calls
After initialization of the DES engine, the library provides a system call which can both encrypt
and decrypt:
int des_cbc_encrypt(clear, cipher, schedule, encrypt)
where the encrypt parameter determines if we are to encipher or decipher. The schedule contains
the secret DES key.
CHAPTER 4. ENCRYPTION AND AUTHENTICATION
36
4.1.4 Case study: Amoeba capabilities
All Amoeba objects are identified by a capability string which is encrypted using DES encryption. A capability is long enough so that you can’t just make them up.
If you have the string, you have whatever the capability allows you. If you want to give someone
some access to a file, you can give them the capability string. They place this in their directory,
and can see the file.
All AMOEBA objects are named/identified by capabilities with four fields:
(48 bits)
Server Port
(24 bits)
(8 bits)
(48 bits)
Object ID
Rights
Checkfield
Identifies the server
which manages the
object
Identifies which
operations are
allowed
Internal number which
the server uses to
identify the object
Protects against forging
To further prevent tampering, the capability is DES encrypted. The resultant bit stream may be
used directly, or converted to and from an ASCII string with the a2c and c2a commands.
4.2 Public key systems
In 1976 Diffie and Hellman published the paper “New Directions in Cryptography”, which first
introduced the idea of public key cryptography. Public key cryptography relies on the use of
enciphering functions which are not realistically invertible unless you have a deciphering key.
These functions are commonly called one-way or trapdoor functions.
4.2.1 Diffie-Hellman key agreement
The Diffie-Hellman paper introduced a new technique which allowed two separated users to
create and share a secret key. A third party listening to all communications between the two
separated users is not realistically able to calculate the shared key.
4.2. PUBLIC KEY SYSTEMS
Alice
p,g,a
37
Bob
gb mod p p,g,b
ga mod p
gb mod p
ga mod p
p,g
gab mod p Ted
g mod p
The knowledge held by each of the participants is different.
• All participants know two system parameters p - a large prime number, and g - an integer
less than p. There are certain constraints on g to ensure that the system is not feasibly
invertible.
• Alice and Bob2 each have a secret value (Alice has a and Bob has b) which they do not
divulge to anyone. Alice and Bob each calculate and exchange a public key (g a mod p for
Alice and g b mod p for Bob).
• Ted knows g, p, g a mod p and g b mod p, but neither a nor b.
Both Alice and Bob can now calculate the value g ab mod p.
1. Alice calculates (g b mod p)a mod p = (g b )a mod p.
2. Bob calculates (g a mod p)b mod p = (g a )b mod p.
And of course (g b )a mod p = (g a )b mod p = g ab mod p - our shared key.
Ted has a much more difficult problem. It is difficult to calculate g ab mod p without knowing
either a or b. The algorithmic run-time of the (so-far best) algorithm for doing this is in
O(ec
√
r log r
)
where c is small, but ≥ 1, and r is the number of bits in the number. By contrast, the enciphering
and deciphering process may be done in O(r):
2
It is common to use the names Bob, Ted, Carol and Alice (from the movie of the same name) when discussing
cryptosystems.
CHAPTER 4. ENCRYPTION AND AUTHENTICATION
38
Bit size
Enciphering
Discrete lograithm solution
10
10
23
100
100
1,386,282
1000
1000
612,700,000,000,000,000,000,000
10000
10000
722,600,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000
Note that we can calculate expressions like g x mod p relatively easily, even when g, x and p
are large. The following code shows an algorithm3 which iterates to a solution, and never has to
calculate a larger number than p2 :
c := 1; { attempting to calculate mod(g Q ,p) }
x := 0;
while x<>Q do
begin
x := x+1;
c := mod(c*g,p)
end;
{ Now c contains mod (g Q ,p) }
4.2.2 Encryption
Public key schemes may be used for encrypting data directly. The transmitter encrypts the signal
using the public key of the recipient.
P
(Plaintext)
K1[P]
P
X
X
K1
K2
(K2[K1[P]]=P)
and also
(K1[K2[P]]=P)
Since the private key may not be generated easily from the public key, the recipient is reasonably
sure that no-one else can decrypt the data.
4.2.3 Authentication
We can use public key schemes to provide authentication. If one machine wants to authentically
transmit information, it encodes using both its private key and the recipient’s public key:
3
Stephen Glasby points out that this is a very slow algorithm. Perhaps you would like to consider how it could
be improved?
4.2. PUBLIC KEY SYSTEMS
P
39
K1[J2[P]]
P
X
X
X
X
J2
K1
K2
J1
The second machine uses the others public key and its own private key to decode.
4.2.4 RSA (Rivest, Shamir, Adelman)
This public key system relies on the difficult problem of trying to find the complete factorization
of a large composite4 integer whose prime factors5 are not known.
Two RSA-encrypted messages have been cracked:
• The inventors of RSA published a message encrypted with a 129-digits (430 bits) RSA
public key6 , and offered $100 to the first person who could decrypt the message. In
1994, an international team coordinated by Paul Leyland, Derek Atkins, Arjen Lenstra,
and Michael Graff successfully factored this public key and recovered the plaintext. The
message read: THE MAGIC WORDS ARE SQUEAMISH OSSIFRAGE.
About 1600 machines took part in the crack, and the project took about eight months and
approximately 5000 MIPS-years of computing time.
• A year later, a 384-bit PGP key was cracked. A team consisting of Alec Muffett, Paul
Leyland, Arjen Lenstra and Jim Gillogly managed to use enough computation power (approximately 1300 MIPS-years) to factor the key in three months. It was then used to
decrypt a publicly-available message encrypted with that key.
Note that these efforts each only cracked a single RSA key.
4.2.5 RSA coding algorithms
Below are outlined the four processes needed for RSA encryption:
1. Creating a public key
4
An integer larger than 1 is called composite if it has at least one divisor larger than 1.
The Fundamental Theorem of Arithmetic states that any integer N (greater than 0) may be expressed uniquely
as the product of prime numbers.
6
For more information on the RSA-129 project, see: ftp://ftp.ox.ac.uk/pub/math/rsa129/rsa129.ps.gz
5
CHAPTER 4. ENCRYPTION AND AUTHENTICATION
40
2. Creating a secret key
3. Encrypting messages
4. Decoding messages
To create public key Kp :
1. Select two different large primes P and Q.
2. Assign x = (P − 1)(Q − 1).
3. Choose E relative prime to x. (This must satisfy condition for Ks given later)
4. Assign N = P ∗ Q.
5. Kp is N concatenated with E.
To create private (secret) key Ks :
1. Choose D: mod(D ∗ E, x) = 1.
2. Ks is N concatenated with D.
We encode plain text P by:
1. Pretend P is a number.
2. Calculate c = mod(P E , N ).
To decode C back to P :
1. Calculate P = mod(C D , N ).
4.2.6 Case study: PGP
PGP (Pretty Good Privacy) is a public key encryption package to protect E-mail and data files.
It lets you communicate securely with people you’ve never met, with no secure channels needed
for prior exchange of keys.
The handout details operation of the UNIX and DOS command line version of PGP, but there are
friendly versions available for UNIX, Win95, WinNT and Macintosh.
Chapter 5
System (in)security
One of my sons was recently taught stranger-danger at his school. We were asked
to quiz him afterwards, so we asked him if he should accept a lift in a car with a
stranger. He immediately replied “No way! ”. We then asked: “What if he offered
you sweets?”, but he still replied “No way! ”. Finally we asked: “Why not? ”, to
which he replied “Because you might not get any !”
5.1 Ethical concerns
A mature ethical sense normally develops as you age. We recognize that young people are
not capable of fully comprehending the world around them. Lawrence Kohlberg, a Harvard
psychologist, formalizes moral development into various stages:
Stage 1: Obedience and punishment - deference to superior power or prestige.
Stage 2: Naively egoistic - a right action satisfying the self’s needs and occasionally others.
Stage 3: Good-boy/good-girl - an orientation to approval, to pleasing and helping others, with
conformity to stereotypical images of majority or natural role behavior.
Stage 4: Authority and social-order-maintaining - an orientation to "doing duty" and to showing
respect for authority and maintaining the given social order for its own sake.
Stage 5: Contractual/legalistic - defined in terms of laws or institutionalized rules.
Stage 6: Individual principles of conscience - an orientation not only toward existing social
rules, but also toward the conscience as a directing agent, mutual trust and respect, and
principles of moral choice involving logical universalities and consistency. If one acts
otherwise, self-condemnation and guilt result.
41
CHAPTER 5. SYSTEM (IN)SECURITY
42
It is my expectation, and requirement, that you are able to maturely evaluate rights and wrongs.
Why? - Because in these later sections of the course, I will be outlining systems which demonstrate poor cryptographic techniques, and as a result, can be defeated.
A more cynical view might be that I am teaching hacking1 . This view is certainly not my intent,
and I only discuss hacks/cracks within a framework of how you can fix it.
5.1.1 Why study this?
Many common views related to insecurity are promoted by the popular press, driven by commercial interests: “Use 128-bit encryption - guaranteed secure”, “Use NT - secure your network”,
and so on. These sort of slogans give a false sense of security, and are commonly incorrect.
An uninformed belief in the safety of computer systems leads to insecure systems, and is reason
enough to study this area. However, if you wish more justifications:
• An awareness of the nature and limits of cryptography is essential to computer people in
business. Computer systems are becoming more critical, and directly relate to the core
operations of many businesses2 .
• Network administrators commonly need to attempt to hack their own systems to assure
themselves of the security of those systems.
• A forgotten encryption password may need to be recovered.
5.1.2 Ethics and computing
In general computer based systems do not introduce any new ethical dilemmas. In most cases it
is relatively easy to draw a parallel with existing non-computer systems. Here are some sample
areas:
Software duplication: It is very easy to duplicate software at no cost. However, doing so can
only be viewed as theft.
Using information: It is often easy to recover information from computer systems - for example
a programmer may become aware of her employer’s proprietary algorithms and then make
use of this knowledge to make money. This is known as insider trading and is considered
a crime.
1
The perjorative term hacking has a proud history - it originally meant “a codesmith”, but has been perverted to
mean “someone who breaks into computer systems”. I prefer to use the term cracker, rather than hacker.
2
It is interesting to note that of those businesses that were unable to restore their computer systems after the San
Francisco earthquake, 50% failed within the next year.
5.1. ETHICAL CONCERNS
43
E-mail abuse: E-mail is no different from any other communication, and most countries already
have laws that inhibit reading, tampering, changing or intercepting mail. Abuse over email
is no different from any other form of (non-contact) abuse.
Network administrator’s dilemma: Network administrators often come to learn things about
their ’clients’ that they did not intend. However, without asking the client, they should not
make use of that information. The documents that most computer users sign when they
are given access to a computer system do not over-ride their legal rights. This leads to the
network administrator’s dilemma: How to control bad-guys without trampling over their
rights.
Perhaps the only significant difference is that the computer crimes are so easy.
5.1.3 Professional codes of ethics
Most professional bodies3 have formal written codes of ethics, along with committees to deal
with abuses of the ethical standards set. The computer industry has yet to develop a single
standard code of conduct, and if computer crime continues to rise, codes may be imposed on it.
The Australian Computer Society proposes a code of ethics which include the following sections:
1. I will serve the interests of my clients and employers, my employees and students, and
the community generally, as matters of no less priority than the interests of myself or my
colleagues.
(a) I will endeavour to preserve continuity of computing services and information flow in
my care.
(b) I will endeavour to preserve the integrity and security of others’ information.
(c) I will respect the proprietary nature of others’ information.
(d) I will advise my client or employer of any potential conflicts of interest between my
assignment and legal or other accepted community requirements.
(e) I will advise my clients and employers as soon as possible of any conflicts of interest
or conscientious objections which face me in connection with my work.
2. I will work competently and diligently for my clients and employers .
(a) I will endeavour to provide products and services which match the operational and
financial needs of my clients and employers.
(b) I will give value for money in the services and products I supply.
(c) I will make myself aware of relevant standards, and act accordingly.
3
For example: Medical boards.
CHAPTER 5. SYSTEM (IN)SECURITY
44
(d) I will respect and protect my clients’ and employers’ proprietary interests.
(e) I will accept responsibility for my work.
(f) I will advise my clients and employers when l believe a proposed project is not in their
best interests.
(g) I will go beyond my brief, if necessary, in order to act professionally.
3. I will be honest in my representations of skills, knowledge, services and products.
(a) I will not knowingly mislead a client or potential client as to the suitability of a product
or service.
(b) I will not misrepresent my skills or knowledge.
(c) I will give opinions which are as far as possible unbiased and objective.
(d) I will give realistic estimates for projects under my control.
(e) I will not give professional opinions which I know are based on limited knowledge or
experience.
(f) l will give credit for work done by others where credit is due.
4. I will strive to enhance the quality of life of those affected by my work.
(a) I will protect and promote the health and safety of those affected by my work.
(b) I will consider and respect people’s privacy which might be affected by my work.
(c) I will respect my employees and refrain from treating them unfairly.
(d) I will endeavour to understand. and give due regard to, the perceptions of those
affected by my work, whether or not I agree with those perceptions.
(e) I will attempt to increase the feelings of personal satisfactions, competence, and
control of those affected by my work.
(f) I will not require, or attempt to influence, any person to take any action which would
involve a breach of this Code.
5. I will enhance my own professional development, and that of my colleagues, employees
and students.
6. I will enhance the integrity of the Computing Profession and the respect of its members for
each other.
Within a general framework of ethical and moral responsibility, codes such as this one can help
clarify grey areas of concern.
5.2. INSECURITY - THREATS AND PROTECTION
45
5.2 Insecurity - threats and protection
The dangers of the use of insecure systems cannot be underestimated. Supposedly secure systems
at the CIA, the Pentagon and the DOD have all been hacked. For example:
• Pentagon machines were repeatedly corrupted by unknown intruders during the Gulf war.
The intruders appeared to be doing it as part of a contest.
• German hackers demonstrated on TV a method of transferring money into their own accounts using ActiveX controls downloaded to an unsuspecting person’s machine.
• Estimates of computer theft in the US range from 1 to 30 $billion/year - most of which
goes unreported.
There have been various attempts to provide a taxonomy of insecurity, but each new attack seems
to add new levels to the structure. We start of course with the obvious:
• physical insecurity, and
• password insecurity
Some of the security of modern systems is provided through cryptographic techniques (particularly password storage), and this course concentrates on these insecurities.
5.2.1 Non-cryptographic cracking
General hacking/cracking is not limited to cryptographic methods, and may often be done much
more quickly. For the sake of completeness, here are some of the general strategies employed in
hack attacks - many can be used either by internal attackers or by remote (external) attackers.
Misconfiguration: If excessive permission exist on certain directories and files, these can lead to
gaining higher levels of access. For example, on a UNIX system, if /dev/kmem is writable
it is possible to rewrite your UID to match root’s.
Poor SUID: Sometimes there are scripts (shell or Perl) that perform certain tasks and run as
root. If the scripts are writable by you, you can edit it and run it.
Buffer overflow: Buffer overflows are typically used to spawn root shells from a process running as root. A buffer overflow could occur when a program has a buffer for user-defined
data and the user-defined data’s length is not checked before the program acts upon it.
CHAPTER 5. SYSTEM (IN)SECURITY
46
Race conditions: A race condition is when a program creates a short opportunity for attack by
opening a small window of vulnerability. For example, a program that alters a sensitive file
might use a temporary backup copy of the file during its alteration. If the permissions on
that temporary file allow it to be edited, it might be possible to alter it before the program
finishes its editing process.
Poor temporary_files: Many programs create temporary files while they run. If a program runs
as root and is not careful about where it puts its temporary files and what permissions these
files have, it might be possible to use links to create root-owned files.
Attacks using these methods can be launched locally on the target machine, or often remotely, by
exploiting services with loopholes. In this context, a service may be a web server, a file server,
an ftp server, or even a security password server.
5.2.2 Protection
Can you protect yourself against attacks? - Yes - but only up to a point. You can reduce your
vulnerability by continual re-examination of your computer systems. The following points are
often made:
• Hack/crack yourself: A common activity of network administrators is to attempt to
hack.crack their own systems, and to encourage friendly colleagues to do the same.
• Be vigilant: There are new exploits discovered every day, and you can keep relatively
up-to-date by subscribing to BugTraq mailing lists.
• Reduce reliance: Don’t rely totally on the security of the machines.
• Use more secure systems: If you are concerned about security, use more secure systems.
Enforce encrypted communications, inhibit plaintext passwords and so on.
• Update systems: More recent revisions of the software normally have better security features.
Finally: “Its not the end of the world!” If your system is damaged, its not the end of the world.
Fix the flaw, fix the damage and get back to work.
5.3. CERT - COMPUTER EMERGENCY RESPONSE TEAM
47
5.3 CERT - Computer Emergency Response Team
CERT describes itself in the following way:
The CERT Coordination Center is the organization that grew from the computer
emergency response team formed by the Defense Advanced Research Projects Agency
(DARPA) in November 1988 in response to the needs identified during the Internet
worm incident. The CERT charter is to work with the Internet community to facilitate
its response to computer security events involving Internet hosts, to take proactive
steps to raise the community’s awareness of computer security issues, and to conduct
research targeted at improving the security of existing systems.
The CERT/CC offers 24-hour technical assistance for responding to computer security incidents, product vulnerability assistance, technical documents, and courses.
In addition, the team maintains a mailing list for CERT advisories, and provides a
web site (www.cert.org) and an anonymous FTP server, (ftp.cert.org) where securityrelated documents, CERT advisories, and tools are available.
The CERT Coordination Center is part of the Networked System Survivability (NSS)
program at the Software Engineering Institute (SEI), a federally funded research and
development center (FFRDC) at Carnegie Mellon University (CMU).
If you are ever involved in a computer security incident it is useful to get in touch with CERT.
They provide incident reports and advisories, and can liaise with other system administration
people if the attack on your system comes from outside your organization.
5.3.1 CERTő Incident Note IN-99-04
Here is an excerpt from a recent incident report:
Similar Attacks Using Various RPC Services
Thursday, July 22, 1999
Overview
We have recently received an increasing number of reports that intruders are using similar methods to compromise systems. We have seen
intruders exploit three different RPC service vulnerabilities; however, similar artifacts have been found on compromised systems.
Vulnerabilities we have seen exploited as a part of these attacks include:
•
CA-99-08 - Buffer Overflow Vulnerability in rpc.cmsd
http://www.cert.org/advisories/CA-99-08-cmsd.html
•
CA-99-05 - Vulnerability in statd exposes vulnerability in automountd
http://www.cert.org/advisories/CA-99-05-statd-automountd.html
•
CA-98.11 - Vulnerability in ToolTalk RPC Service
http://www.cert.org/advisories/CA-98.11.tooltalk.html
CHAPTER 5. SYSTEM (IN)SECURITY
48
Description
Recent reports involving these vulnerabilities have involved very similar intruder activity. The level of activity and the scope of the incidents
suggests that intruders are using scripts to automate attacks. These attacks appear to attempt multiple exploitations but produce similar results.
We have received reports of the following types of activity associated with these attacks:
•
•
•
Core files for rpc.ttdbserverd located in the root "/" directory, left by an exploitation attempt against rpc.ttdbserverd
Files named callog.* located in the cmsd spool directory, left by an exploitation attempt against rpc.cmsd
Exploitations that execute similar commands to create a privileged back door into a compromised host. Typically, a second instance
of the inetd daemon is started using an intruder-supplied configuration file. The configuration file commonly contains an entry that
provides the intruder a privileged back door into the compromised host. The most common example we have seen looks like this:
/bin/sh -c echo ’ingreslock stream tcp wait root /bin/sh -i’ >> /tmp/bob;/usr/sbin/inetd -s /tmp/bob
If successfully installed and executed, this back door may be used by an intruder to gain privileged (e.g., root) access to a compromised host by connecting to the port associated with the ingreslock service, which is typically TCP port 1524. The file names and
service names are arbitrary; they may be changed to create an inetd configuration file in a different location or a back door on a different
port.
•
In many cases, scripts have been used to automate intruder exploitation of back doors installed on compromised hosts. This method has
been used to install and execute various intruder tools and tool archives, initiate attacks on other hosts, and collect output from intruder
tools such as packet sniffers.
One common set of intruder tools we have seen is included in an archive file called neet.tar, which includes several intruder tools:
–
–
–
•
A packet sniffer named update or update.hme that produces an output file named output or output.hme
A back door program named doc that is installed as a replacement to /usr/sbin/inetd. The back door is activated when a
connection is received from a particular source port and a special string is provided. We have seen the source port of 53982
commonly used.
A replacement ps program to hide intruder processes. We have seen a configuration file installed at /tmp/ps_data on compromised hosts.
Another common set of intruder tools we have seen is included in an archive file called leaf.tar, which includes serveral intruder tools:
–
–
A replacement in.fingerd program with a back door for intruder access to the compromised host
–
–
Various files and scripts associated with eggdrop, many of which are installed in the directory /usr/lib/rel.so.1
eggdrop, an IRC tool commonly installed on compromised hosts by intruders. In this activity, we’ve seen the binary installed
as /usr/sbin/nfds
A replacement root crontab entry used to start eggdrop
It is possible that other tools and tool archives could be involved in similar activity.
In some cases, we have seen intruder scripts remove or destroy system binaries and configuration files.
5.4. NSA - NATIONAL SECURITY AGENCY
49
5.4 NSA - National Security Agency
NSA describes itself in the following way:
The National Security Agency is the USA’s cryptologic organization. It coordinates,
directs, and performs highly specialized activities to protect U.S. information systems and produce foreign intelligence information. A high technology organization,
NSA is on the frontiers of communications and data processing. It is also one of the
most important centers of foreign language analysis and research within the Government.
Signals Intelligence (SIGINT) is a unique discipline with a long and storied past.
SIGINT’s modern era dates to World War II, when the U.S. broke the Japanese military code and learned of plans to invade Midway Island. This intelligence allowed
the U.S. to defeat Japan’s superior fleet. The use of SIGINT is believed to have
directly contributed to shortening the war by at least one year. Today, SIGINT continues to play an important role in maintaining the superpower status of the United
States.
NSA employs the country’s premier codemakers and codebreakers. It is said to be
the largest employer of mathematicians in the United States and perhaps the world.
Its mathematicians contribute directly to the two missions of the Agency: designing cipher systems that will protect the integrity of U.S. information systems and
searching for weaknesses in adversaries’ systems and codes.
In 1943, SIGINT, a forerunner of the National Security Agency, began a very secret program,
codenamed VENONA. The object of the VENONA program was to examine encrypted Soviet
diplomatic communications. In October 1943, weaknesses were discovered in the cryptographic
system of the Soviet trade traffic.
During 1944, the skills of other expert cryptanalysts were brought to bear on the message traffic
to see if any of the encryption systems of the messages could be broken. One of these cryptanalysts made observations which led to a fundamental break into the cipher system used by the
KGB. The messages were double-encrypted and were extremely difficult to crack. It took almost
two more years before parts of any of these KGB messages could be read or even be recognized
as KGB rather than standard diplomatic communications.
Almost all of the KGB messages between Moscow and New York, and Moscow and Washington
in 1944 and 1945 that could be broken at all were broken between 1947 and 1952.
NSA continue this sort of work actively, but more recent work is classified.
CHAPTER 5. SYSTEM (IN)SECURITY
50
5.5 C2 security
The NSA created various criteria for evaluating the security behaviour of machines. These criteria were published in a series of documents with brightly coloured covers, and hence became
known as the Rainbow series.
The document DOD 5200.28-STD4 - “Department of Defense Trusted Computer System Evaluation Criteria”, has been developed to serve a number of purposes:
• To provide a standard to manufacturers as to what security features to build into their
new and planned, commercial products in order to provide widely available systems that
satisfy trust requirements (with particular emphasis on preventing the disclosure of data)
for sensitive applications.
• To provide DoD components with a metric with which to evaluate the degree of trust
that can be placed in computer systems for the secure processing of classified and other
sensitive information.
• To provide a basis for specifying security requirements in acquisition specifications.
The term C2 comes from these documents, and describes a set of desireable security features
related to controlled access, that were considered by the US Department of Defense when the
documents were developed. Many of the elements of a C2-secure, system are just those functions
that should not be enabled on a system, and so making a system C2-secure includes turning off
some features of a system.
For example, C2 requires that:
The TCB5 shall require users to identify themselves to it before beginning to perform any other actions that the TCB is expected to mediate. Furthermore, the TCB
shall use a protected mechanism (e.g., passwords) to authenticate the user’s identity. The TCB shall protect authentication data so that it cannot be accessed by any
unauthorized user. The TCB shall be able to enforce individual accountability by
providing the capability to uniquely identify each individual ADP system user. The
TCB shall also provide the capability of associating this identity with all auditable
actions taken by that individual.
And so on...
Windows NT Workstation vs 3.5 with U.S. Service Pack 3 is the only Microsoft product that has
completed C2 testing, and is only certified if using the same hardware, and installed software,
4
This document may be found at http://www.radium.ncsc.mil/tpep/library/rainbow/5200.28-STD.html, and a
local copy is kept at http://opo.usp.ac.fj/˜hugh/Public/cs492/extradocs/5200.28-STD.html.
5
Trusted Computing Base.
5.5. C2 SECURITY
51
and does not include any network connection. The NT utility c2config.exe sets up an NT system
to pass the C2 tests. Many UNIX systems have also got C2 certification, and come configured
this way from the manufacturer.
The 1998 attacks on the Pentagon involved theft and modification of data, as well as
denial-of-service. The attacked machines were C2-secure Windows NT machines.
Many UNIX systems have also got C2 certification, and come configured this way from the
manufacturer.
There are numerous examples of hacked UNIX systems found on the Internet. In
1996, a site I managed in New Zealand was the target of a malicious attack by
intruders from Australia and Belgium.
Given all this, C2 certification is probably not a good guide as to the security of your system.
52
CHAPTER 5. SYSTEM (IN)SECURITY
Chapter 6
Insecurity case studies
6.1 PkZip stream cipher
PkZip is a shareware utility for compressing and encrypting files. It has been available for many
years, and is responsible for the zip extension found on many files. Most other compression or
archiving utilities provide some level of compatability with PkZip’s compression scheme.
PkZip can also scramble files when given a secret password. However, the enciphering strategy
is weak and can be cracked using a known-plaintext style of attack. Three 32-bit keys are generated from the original enciphering text, and the resultant 96-bit code is the core of the stream
cipher algorithm. The stream cipher algorithm, and method of attack is described in Niham and
Kocher’s paper1 “A Known Plaintext Attack on the PKZIP Stream Cipher”. The attack exploits
a weakness in the (homegrown) ciphering algorithm, which allows us to collect possible values
for one of the keys, discarding impossible values, and then use those possible values to calculate
the other keys.
Here we see the attack in use, extracting the keys for a zipped and encrypted archive all.zip, with
known plaintext readme.doc also available in zipped form in the file plain.zip:
opo 144% pkcrack -C all.zip -c readme.doc -P plain.zip -p readme.doc
Files read. Starting stage 1 on Wed Sep 8 09:04:02 1999
Generating 1st generation of possible key2_421 values...done.
Found 4194304 possible key2-values.
Now we’re trying to reduce these...
Done. Left with 18637 possible Values. bestOffset is 24.
Stage 1 completed. Starting stage 2 on Thu Sep 9 09:12:06 1999
Ta-daaaaa! key0=dda9e469, key1=96212999, key2=f9fc9651
Probabilistic test succeeded for 402 bytes.
Stage2 completed. Starting password search on Thu Sep 9 09:22:22 1999
Key: 73 65 63 72 65 74
Or as a string: ’secret’ (without the enclosing single quotes)
Finished on Thu Sep 9 10:54:22 1999 opo 99%
opo 145% ./zipdecrypt dda9e469 96212999 f9fc9651 all.zip rr.zip
opo 146%
At the completion of the above commands, rr.zip contains an unencypted version of all the files
in the original archive.
1
A copy is held at http://opo.usp.ac.fj/˜hugh/Public/cs492/extradocs/pkzip.ps.gz.
53
CHAPTER 6. INSECURITY CASE STUDIES
54
6.1.1 PkZip stream cipher fix
The PkZip stream cipher is also susceptible to dictionary attacks, and so it is considered not
suitable for secure encryption of data. The fix is:
Don’t use PkZip for security purposes.
6.2 UNIX base security
UNIX systems are traditionally open systems, given their background in university environments. As such, the security on them is often minimal. It is common for UNIX accounts to be
made available relatively freely. For example, at the MIT Media lab2 all computers have been
password-free until recently - an expression of academic freedom.
UNIX systems are vulnerable to a wide range of attacks, particularly internal attacks (where the
attacker has an existing account). However in the context of this course, we will look at the
UNIX account system.
All Unix systems have a root account. This account has a UID and GID of zero, and once root
access is obtained on a UNIX system, there is very little that cannot be done. Account passwords
are constructed to meet the following requirements:
• Each password has at least six characters.
• Only the first eight characters are significant.
There are many other accounts found on Unix systems, not just those for clients. For example
the following accounts are commonly found:
sysadm - A System V administration account, and
daemon - A daemon process account, and
uucp - The UUCP owner, and
lp - The print spooler owner.
Account information is kept in a file called /etc/passwd. It normally consists of seven colondelimited fields, which may look like the following:
hugo:aAbBcJJJx23F55:501:100:Hughs Account:/home/hugo:/bin/tcsh
2
Yes - thats right - MIT - home of perhaps the most secure password system in use today.
6.2. UNIX BASE SECURITY
55
The fields are:
hugo: The account or user name.
aAbBcJJJx23F55: A one-way encrypted (hashed) password, and optional password aging information.
501: The UID - Unique user number
100: The GID - Group number of the default group that the user belongs to.
Hughs Account: Account information. In some versions of UNIX, this field also
contains the user’s office, extension, home phone, and so on. For historical reasons
this field is called the GECOS field.
/home/hugo: The account’s home directory
/bin/tcsh: A program to run when you log in - usually a shell
UNIX uses a DES-like algorithm to calculate the encrypted password. The password is used
as the DES key (eight 7-bit characters make a 56 bit DES key) to encrypt a block of binary
zeroes. The result of this encryption is the hash value. Note: the password is not encrypted, it
is the key used to perform the encryption! A strong feature of UNIX is that it introduces two
random characters in the alogrithm (the salt). This ensures that two equal passwords result in two
different hash values. From viewing the UNIX password file you can not deduce whether two
persons have the same password. Even if they do, the salt introduced in the algorithm ensures
that the hash values will be different.
When you log in with your account name and password, the password is encrypted and the
resulting hash is compared to the hash stored in the password file. If they are equal, the system
accepts that you’ve typed in the correct password and grants you access.
6.2.1 Crypt algorithm
Sample crypt code in java is found at http://opo.usp.ac.fj/˜hugh/Public/cs492/software/crypt.html.
The code has the following structure:
char *crypt(char *buf; char *salt)
{
/* Construct the salt values */
.....
/* Make key */
.....
des_set_key((des_cblock *)(key),ks);
.....
/* permute the string */
.....
/* return the permuted string */
buff[13]=’\0’;
return((char *)buff);
}
CHAPTER 6. INSECURITY CASE STUDIES
56
To prevent crackers from simply encrypting an entire dictionary and then looking up the hash,
the salt was added to the algorithm to create a possible 4096 different conceivable hashes for a
particular password. This lengthens the cracking time because it becomes a little harder to store
an encrypted dictionary online as the encrypted dictionary now would have to take up 4096 times
the disk space. This does not make password cracking harder, just more time consuming.
6.2.2 Brute force cracking
Brute force password cracking is simply trying a password of A with the given salt, folowing by
B, C, and on and on until every possible character combination is tried. It is very time consuming,
but given enough time, brute force cracking will get the password.
The hashed passwords are compared with the entry in the /etc/passwd file. There is no utility
in trying to exhaustively log in using all the possible passwords, as UNIX systems enforce 10
second timeouts after three consecutive login failures.
6.2.3 Dictionary cracking
Dictionary password cracking is the most popular method for cracking Unix passwords. The
cracking program will take a word list, and one at a time try to crack one or all of the passwords
listed in the password file. Some password crackers will filter and/or mutate the words as they
try them, such as substitute numbers for certain letters, add prefixes or suffixes, or switch case or
order of letters.
A popular cracking utility is called crack 3 . It can be configured by an administrator to periodically run and send email to users with weak passwords, or it may be run in manual mode. Crack
can also be configured to run across multiple systems and to use user-definable rules for word
manipulation/mutation to maximize dictionary effectiveness.
6.2.4 UNIX base security fix
The susceptibility of UNIX systems to dictionary attacks has been known for many years, and
a system known as shadow passwords is used to fix the problem. Most modern UNIXes either
use shadow passwords out-of-the-box, or can be configured to use them by running a utility. On
IRIX, the pwconv utility is described as follows:
The pwconv command creates and updates /etc/shadow with information from /etc/passwd.
If the /etc/shadow file does not exist, pwconv creates /etc/shadow with information from
/etc/passwd. The command populates /etc/shadow with the user’s login name, password,
and password aging information. If password aging information does not exist in /etc/passwd
3
Crack may be found at http://opo.usp.ac.fj/˜hugh/Public/cs492/software/crack5.0.tar.gz.
6.3. MICROSOFT BASE SECURITY
57
for a given user, none is added to /etc/shadow. However, the last changed information is
always updated.
The pwconv program is a privileged system command that cannot be executed by ordinary
users.
Once the password hashes are moved to the shadow file, its permissions are changed as follows:
opo 35# ls -l /etc/shadow
-r-------1 root
sys
opo 36#
3429 Aug 20 14:46 /etc/shadow
These permissions ensure that ordinary users are unable to look at the password hashes, and
hence are unable to try dictionary attacks.
6.3 Microsoft base security
Two one-way password hashes are stored on NT systems:
• a LanManager hash, and
• a Windows NT hash.
The LanManager hash supports the older LanManager protocol originally used in Windows and
OS/2. In an all-NT environment it is desirable to turn off LanManager passwords, as it is easier
to crack. The NT method uses a stronger algorithm and allows mixed-cased passwords.
The database containing these hashes on an NT system is called the SAM (Security Access
Manager) database and is used for all user authentication as well as inter-process authentication.
If you have administrative access4 , the program pwdump can extract the hashes. The hashes may
also be directly captured from a local area network using a sniff utility such as readsmb 5 .
In figure 6.1, we see network login traffic between a Windows client and a PDC (Primary Domain
Controller). If the network media is a shared bus (such as ethernet), then the login traffic may be
sniffed (or snooped) by a third party.
Microsoft does not salt during hash generation, so once a potential password has generated a
hash it can be checked against all accounts. The crack software takes advantage of this.
4
Originally, anyone could extract the hashed passwords from the SAM, as Microsoft believed that “if they didn’t
tell anyone the algorithms they used, no-one could discover what they had done”. Security through obscurity is not
a safe strategy, and Jeremy Allison was able to de-obfuscate the SAM entries relatively quickly.
5
The security strategies used by Microsoft have been uncovered by the SAMBA team, to allow their development
of an open source SMB file and print service. Some parts of this section have been extracted from the SAMBA
documentation.
CHAPTER 6. INSECURITY CASE STUDIES
58
PDC
CLIENT
Login network traffic
Snooping!
BAD GUY!
Figure 6.1: Network login traffic snooping.
6.3.1 LanManager encryption
LanManager encryption is created by taking the user’s plaintext password, capitalising it, and
either truncating to 14 bytes, or padding to 14 bytes with null bytes. This 14 byte value is used
as two 56-bit DES keys to encrypt a magic eight byte value, forming a 16 byte value which is
stored by the server and client. This value is known as the hashed password .
6.3.2 NT encryption
Windows NT encryption is a higher quality mechanism, consisting of doing an MD4 hash on a
Unicode version of the user’s password. This also produces a 16 byte hash value that is nonreversible.
MD4 is a one-way hashing function developed by Ron Rivest (The R in RSA). It takes 512-bit
blocks as input and outputs a 128-bit fingerprint of the input data. It is described in rfc1320 6 ,
complete with source code detailing the algorithm.
6.3.3 Challenge-response protocol
When a client wishes to use an SMB resource, it first requests a connection and negotiates the
protocol that the client and server will use. In the reply to this request the server generates and
appends an 8 byte, random value - this is stored in the server after the reply is sent and is known
as the challenge . It is different for every client connection.
The client then uses the hashed password (16 byte values described above), appended with 5 null
bytes, as three 56 bit DES keys, each of which is used to encrypt the challenge 8 byte value,
6
A copy is held at http://opo.usp.ac.fj/˜hugh/Public/cs492/extradocs/rfc1320.html.
6.3. MICROSOFT BASE SECURITY
59
forming a 24 byte value known as the response. This calculation is done on both hashes of the
user’s password, and both responses are returned to the server, giving two 24 byte values.
The server then reproduces the above calculation, using its own value of the 16 byte hashed
password and the challenge value that it kept during the initial protocol negotiation. It then
checks to see if the 24 byte value it calculates matches the 24 byte value returned to it from the
client. If these values match exactly, then the client knew the correct password and is allowed
access. If not then the client did not know the correct password and is denied access.
There are good points about this:
• The server never knows or stores the cleartext of the users password - just the 16 byte
hashed values derived from it.
• The cleartext password or 16 byte hashed values are never transmitted over the network thus increasing security.
However, there is also a bad side:
• The 16 byte hashed values are a "password equivalent". You cannot derive the users password from them, but they can be used in a modified client to gain access to a server.
• The initial protocol negotiation is generally insecure, and can be hijacked in a range of
ways. One common hijack involves convincing the server to allow clear-text passwords.
Despite functionality added to NT to protect unauthorized access to the SAM, the mechanism is
trivially insecure - both the hashed values can be retrieved using the network sniffer mentioned
before, and they are as-good-as passwords.
6.3.4 Attack
The security of NT systems relies on a flawed mechanism. Even without network access, it
is possible by various means to access the SAM password hashes, and with network access it is
easy. The hashed values are password equivalents, and may be used directly if you have modified
client software.
The attack considered here is the use of either a dictionary, or brute force attack directly on the
password hashes (which must be first collected somehow).
L0phtCrack is a tool for turning Microsoft Lan Manager and NT password hashes back into the
original clear text passwords. It may be configured to run in different ways.
Dictionary cracking: L0phtCrack running on a Pentium Pro 200 checked a password file with
100 passwords against a 8 Megabyte (about 1,000,000 word) dictionary file in under one
minute.
CHAPTER 6. INSECURITY CASE STUDIES
60
Brute force: L0phtCrack running on a Pentium Pro 200 checked a password file with 10 passwords using the alpha character set (A-Z) in 26 hours.
As the character sets increase in size from 26 characters to 68 the time to brute force the password
increases exponentially. This chart illustrates the relative time for larger character sets.
Character set size
26
36
46
68
Size of computation
9
8.353 ∗ 10
8.060 ∗ 1010
4.455 ∗ 1011
6.823 ∗ 1012
Relative time taken
1.00
9.65
53.33
816.86
So if 26 characters takes 26 hours to complete, a worst-case scenario for 36 characters (A-Z,09) would take 250 hours or 10.5 days. A password such as take2asp1r1n would probably be
computed in about 7 days.
6.3.5 Microsoft base security fix
A range of steps may be taken to reduce exposure due to the hash insecurity.
• Disable the use of Lan Manager passwords.
• Don’t log in over network as any user you do not wish to compromise.
• Encrypt all network traffic (to be discussed later in the section on use of ssh).
• Use long passwords, and all allowable characters, to slow down the crack.
• Use an alternative login system (PAM supports multiple login methods, and there are more
secure systems).
• Use an unsniffable network cabling system.
Chapter 7
Security systems
In this chapter, we look at various systems for securing modern networked computer
systems.
7.1 Kerberos
Kerberos is a network authentication protocol. It is designed to provide strong authentication
for client/server applications by using public key cryptography. Kerberos is freely available
from MIT, in a similar way that they provide the X11 window system. MIT provides Kerberos
in source form, so that anyone who wishes may look over the code for themselves and assure
themselves that the code is trustworthy. Kerberos is also available in commercial products.
The Kerberos protocol uses strong cryptography so that a client can prove its identity to a server
(and vice versa) across an insecure network connection. After a client and server have used
Kerberos to prove their identity, they can also encrypt all of their communications to assure
privacy and data integrity as they go about their business.
KDC
Authentication
Ticket granting
Server
(2) (3)
(1)
(4)
(5)
(6)
Client
When a client first authenticates to Kerberos, she:
61
CHAPTER 7. SECURITY SYSTEMS
62
1. talks to the Authentication Service on the KDC, to ...
2. get a Ticket Granting Ticket (encrypted with the client’s password).
3. When the client wants to talk to a Kerberized service, she uses the Ticket Granting Ticket
to talk to the Ticket Granting Service (which also runs on the KDC). The Ticket Granting
Service verifies the client’s identity using the Ticket Granting Ticket and ...
4. issues a ticket for the desired service.
5. The client may then use the ticket, to...
6. interact with the server.
The reason the Ticket Granting Ticket exists is so a user doesn’t have to enter in their password
every time they wish to connect to a Kerberized service or keep a copy of their password around.
If the Ticket Granting Ticket is compromised, an attacker can only masquerade as a user until
the ticket expires.
7.1.1 Kerberos protocol
Kerberos uses a variant of the Needham-Schroeder protocol described in [2]. There are two sorts
of credentials used, tickets and authenticators .
A ticket Tc,s contains the client’s name and network address, the server’s name, a timestamp and
a session key. This is encrypted with the server’s secret key (so that the client is unable to modify
it).
An authenticator Ac,s contains the client’s name, a timestamp and an optional extra session key.
This is encrypted with the session key shared between the client and the server.
A key Kx,y is a session key shared by both x and y. When we encrypt a message M using the
key Kx,y we write it as {M }Kx,y .
If Alice and Bob both share keys with a trustee (Ted), and Alice wants to get a session key for
communication with Bob, we use the following sequence.
• Alice sends a message to Ted containing her own identity, Ted’s TGS identity, and a onetime value (n) : {a, tgs, n}.
• Ted responds with a key encrypted with Alice’s secret key (which Ted knows), and a ticket
encrypted with the TGS secret key: {Ka,tgs , n}Ka {Ta,tgs }Ktgs .
Alice now has an initial (encrypted) ticket, and a session key: ({Ta,tgs }Ktgs and Ka,tgs ).
• Alice can now prove her identity to the TGS, because she has a session key Ka,tgs , and the
Ticket Granting Ticket: {Ta,tgs }Ktgs .
7.2. SSH
63
Later, Alice can ask the TGS for a specific service ticket:
• When Alice wants a ticket for a specific service (say with Bob), she sends an authenticator
along with the Ticket Granting Ticket to the TGS: {Aa,b }Ka,tgs {Ta,tgs }Ktgs , b, n.
• The TGS responds with a suitable key and a ticket: {Ka,b , n}Ka,tgs {Ta,b }Kb .
• Alice can now use an authenticator and ticket directly with Bob: {Aa,b }Ka,b {Ta,b }Kb .
7.1.2 Weaknesses
Host security: Kerberos makes no provisions for host security; it assumes that it is running
on trusted hosts with an untrusted network. If your host security is compromised, then
Kerberos is compromised as well. If an attacker breaks into a multi-user machine and
steals all of the tickets stored on that machine, he can impersonate the users who have
tickets stored on that machine, but only until those tickets expire.
KDC compromises: Kerberos uses a principal’s password (encryption key) as the fundamental
proof of identity. If a user’s Kerberos password is stolen by an attacker, then the attacker
can impersonate that user with impunity. Since the KDC holds all of the passwords for all
of the principals in a realm, if host security on the KDC is compromised, then the entire
realm is compromised.
Salt: This is an additional input to the one-way hash algorithm. If a salt is supplied, it is concatenated to the plaintext password and the resulting string is converted using the one-way
hash algorithm. In Kerberos 4, a salt was never used. The password was the only input to
the one-way hash function. This has a serious disadvantage; if a user happens to use the
same password in two Kerberos realms, a key compromise in one realm would result in a
key compromise in the other realm.
In Kerberos 5 the complete principal name (including the realm) is used as the salt. This
means that the same password will not result in the same encryption key in different realms
or with two different principals in the same realm. The MIT Kerberos 5 KDC stores the
key salt algorithm along with the principal name, and that is passed back to the client as
part of the authentication exchange.
7.2 ssh
Secure shell (ssh) is a program for logging into a remote machine and for executing commands
in a remote machine. It provides for secure encrypted communications between two untrusted
hosts over an insecure network.
In other words:
CHAPTER 7. SECURITY SYSTEMS
64
• You can’t snoop or sniff passwords.
X11 connections and arbitrary TCP/IP ports can also be forwarded over the secure channel.
The ssh program connects and logs into a specified host. There are various methods that may be
used to prove your identity to the remote machine:
1. /etc/hosts.equiv: If the machine from which the user logs in is listed in /etc/hosts.equiv on
the remote machine, and the user names are the same on both sides, the user is immediately
permitted to log in.
2. ~/.rhosts: If ~/.rhosts or ~/.shosts exists on the remote machine and contains a line containing the name of the client machine and the name of the user on that machine, the user
is permitted to log in.
3. RSA: As a third authentication method, ssh supports RSA based authentication. The
scheme is based on public-key cryptography.
4. TIS: The ssh program asks a trusted server to authenticate the user.
5. Passwords: If other authentication methods fail, ssh prompts the user for a password. The
password is sent to the remote host for checking; however, since all communications are
encrypted, the password cannot be seen by someone listening on the network.
When the user’s identity has been accepted by the server, the server either executes the given
command, or logs into the machine and gives the user a normal shell on the remote machine. All
following communication with the remote command or shell will be automatically encrypted.
7.2.1 RSA key management
Perhaps the most secure part of ssh is its use of RSA key pairs for authentication. The file
~/.ssh/authorized_keys lists the public keys that are permitted for logging in. The RSA login
protocol is:
• Initially: When the user logs in, the ssh program tells the server which key pair it would
like to use for authentication.
• Challenge: The server checks if this key is permitted, and if so, sends the user (actually the
ssh program running on behalf of the user) a challenge and a random number, encrypted
with the user’s public key.
• Decrypt: The challenge can only be decrypted using the proper private key. The user’s
client then decrypts the challenge using the private key. The challenge may be returned in
later (encrypted) messages as proof that the client is valid.
7.2. SSH
65
The user creates an RSA key pair by using the program ssh-keygen . This stores the private key in
~/.ssh/identity and the public key in ~/.ssh/identity.pub. The user can then copy the identity.pub
to .ssh/authorized_keys in his/her home directory on the remote machine (the authorized_keys
file corresponds to the conventional ~/.rhosts file, and has one key per line, though the lines can
be very long). After this, the user can log in without giving the password.
RSA authentication is much more secure than rhosts authentication.
7.2.2 Port forwarding
Secure shell supports TCP/IP port forwarding to connect arbitrary, otherwise insecure connections over a secure channel.
TCP/IP port forwarding works by creating a local proxy server for any desired remote TCP/IP
service. The local proxy server waits for a connection from a client, forwards the request and the
data over the secure channel, and makes the connection to the specified remote service on the
other side of the secure channel.
Proxies can be created for most of the remote services that use TCP/IP. This includes client-server
applications, normal UNIX services like smtp, pop, http, and many others.
[diagram]
The server then listens for connections on this port, forwards the connection request and any data
over the secure channel, and makes a connection to the real X server from the SSH Terminal.
The DISPLAY variable is automatically set to point to the proper value. Note that forwarding
can be chained, permitting safe use of X applications over an arbitrary chain of SSH connections.
7.2.3 Summary
• proxy servers and support for secure X11 connections:
• Proxy servers can be created for arbitrary TCP/IP based remote services and the connections can be forwarded across an insecure network.
• Automatic forwarding for the X11 Windowing System commonly used on UNIX machines.
• CPU overhead caused by strong encryption is of no consequence when transmitting confidential information.
• The strongest available encryption methods should be used, as they are no more expensive
than weak methods.
• Due to compression of transferred data SSH protocol can substantially speed up longdistance transmissions.
CHAPTER 7. SECURITY SYSTEMS
66
7.3 SSL
Netscape has designed and specified a protocol for providing data security layered between application protocols (such as HTTP, Telnet, NNTP, or FTP) and TCP/IP. It uses 128-bit keys.
This security protocol, called Secure Sockets Layer (SSL), provides data encryption, server authentication, message integrity, and optional client authentication for a TCP/IP connection.
SSL is an open, nonproprietary protocol. It has been submitted to the W3 Consortium (W3C)
working group on security for consideration as a standard security approach for World Wide Web
browsers and servers on the Internet.
7.3.1 UN-SSL
Unfortunately, soon after Netscape developed and implemented SSL, a loophole in Netscape’s
own implementation of SSL was discovered.
Netscape seeds a random number generator it uses to produce challenges and master keys with
a combination of the time in seconds and microseconds, and the PID. Of these, only the time in
microseconds is hard to determine by someone who can watch your packets on the network and
has access to any account on the system running netscape.
Even if you do not have an account on the system running netscape, the time can often be
obtained from the time or daytime network daemons. The PID can sometimes be obtained from
a mail daemon. Clever guessing of these values cuts the expected search space down to less than
brute-forcing a 40-bit key, and certainly is less than brute-forcing a 128-bit key.
Due to these poor implmentation decisions, software which can successfully snoop on the original implementation of SSL has been available for some time.
7.4 PGPfone
PGPfone1 lets you whisper in someone’s ear, even if their ear is a thousand miles away. PGPfone
(Pretty Good Privacy Phone) is a software package that turns your desktop or notebook computer
into a secure telephone:
1
(From the documentation)
7.4. PGPFONE
67
It uses speech compression and strong cryptographic protocols to give you the ability to have
a real time secure telephone conversation. PGPfone takes your voice from a microphone, then
continuously digitizes, compresses and encrypts it and sends it out to the person at the other end
who is also running PGPfone.
All cryptographic and speech compression protocols are negotiated dynamically and invisibly,
providing a natural user interface similar to using a normal telephone. Public key protocols are
used to negotiate keys. Enough advertising!
One of the peculiarities about PGPfone, is that it is available in two versions:
1. An international version available outside America, and a prohibited import into America.
2. An American version available inside America, and a prohibited import out of America.
These two versions are also exactly the same! This peculiar situation is a result of American
restrictions on the import and export of munitions - strong cryptography is considered a munition.
When we look at the preferences dialog, we see familiar encryption and key exchange parameters:
When initially setting up a link, Diffie-Hellman key exchange is used to ensure safety in the
choice of an encryption key. The handout summarizes the messages.
68
CHAPTER 7. SECURITY SYSTEMS
Bibliography
[1] C.E. Shannon, A mathematical Theory of Communication, “The Bell System Technical Journal”, Vol 27, pp. 379-423, 623-656, July, October 1948.
[2] Needham Schroeder (ref?)
69
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement