UNIT V

Security at the Application Layer:

Security at the Network Layer:
PGP and S/MIME:
IPSec:
Two modes,
Security at the Transport Layer:
Two security protocols,
SSL and TLS:
Security association,
SSL Architecture,
security policy,
Four Protocols,
Internet Key exchange,
SSL Message Formats,
ISAKMP.
Transport Layer Security.
Security at the Application Layer:

In today’s electronic world, email is critical to any business being competitive. In

most cases it now forms the backbone of most organization's. day-to-day activities,
and its use will continue to grow.

E-mail communication involves sending/ transferring and receiving. In Internet e-

mail communication, some standard methods (protocols) are used for Sending
/transferring and receiving e-mail so that involving software can understand each
other.

In Internet e-mail communication, the standard/protocol used for

sending/transferring email is SMTP (Simple Mail Transfer Protocol), while the
standards/protocols used for receiving e-mail are POP (Post Office Protocol) and
IMAP (Internet Message Access Protocol).
E-mail Threats
The widespread use of email has provided hackers and crackers with an easy way to
distribute harmful content to the internal network.

Hackers can easily circumvent the protection offered by a firewall by tunneling through
the email protocol, since it does not analyze email content.

Furthermore, email is also used to install Trojans, targeted specifically at your

organization to obtain confidential information or gain control of your servers.

Described as instructive viruses or spy viruses by computer security experts, these

can be potent tools in industrial espionage(the systematic use of spies to get secrets).
There are two schemes used secure email information PGP and S/MIME.
PGP (Pretty Good Privacy)
PGP is a computer program that provides
cryptographic privacy and authentication.
PGP is often used for signing,
Encrypting/decrypting emails to increase the
security of email communication.
It was originally created by Phillip Zimmermann
in 1991.
PGP encryption uses public-key cryptography
and includes a system that bind the public
keys to a user name and/or an email address.
PGP supports message authentication
and integrity checking.
For digital signature creation RSA or
DSS algorithm is used.
It is available free world wide and
run on a variety of platforms.
PGP supports symmetric encryption
also. Uses algorithms like CAST-128,
IDEA, TDES.
For Hashing SHA-1 is used.
Notations used in the processes of PGP are as follows:

Ks Session key used in conventional encryption

KRa Private key of user A, used in PK encryption
KUa Public key of user A , used in PK encryption
EP Public-key encryption
DP Conventional decryption
EC Conventional encryption
DC Conventionall decryption
H Hash function
|| Concatenation
Z compression using ZIP algorithm
R64 Conversion to radix 64 ASCII format
PGP offers 5 services:
Authentication
Confidentiality
Compression
Email compatibility
Segmentation
PGP Authentication
In PGP authentication is service is provided through digital signatures. The Sequence as
follows:
 The sender creates message
 SHA-1 is used to generate a 160-bit hash code of the message.
 The Hash code is encrypted with RSA using the sender’s private key, and the result is
prepended to the message.
 The receiver uses RSA with the sender’s public key to decrypt and recover the
 hash code.
 The receiver generates a new hash code for the message and compares it with the
decrypted hash code. If the two match, the message is accepted as authentic.
 The signatures are also generated using DSS/SHA-1 instead of RSA/SHA-1. For
additional security and efficient transmission ZIP algorithms are also used.
PGP Confidentiality
Confidentiality or the privacy of the message may be maintained in PGP by the
conventional encryption algorithm CAST-128 and alternatively IDEA or TDEA. The 64-
bit cipher feedback(CFB) mode is used.
Both PK and also SK encryptions are used. The key distribution is the main concern here.
The one-time conventional key (session key) is generated for each message and
transmitted along with the message.
The illustration is as follows:
 The sender generates a message and a session key
 The message is encrypted, using conventional algorithm with session key.
 The session key is encrypted with RSA, using the recipient’s public key and
prepended to the message.
 The receiver uses RSA with its private key to decrypt and recover the session
key.
 The session key is used to decrypt the message.
Compression
PGP compresses the message after applying the signature but before encryption, i.e. when
message authentication is required the compression is done after signing the
message and when message confidentiality is required, compression is done before
encryption.

This benefits in saving space both for storage as well as for transmission of
email message.
The signature is generated before compression for reasons below :
In order to store uncompressed message together with the signature for future
verification. If it would have had compressed before signing then later verification is difficult.
Because PGP.s compression algorithm is not deterministic; various implementations achieve
different tradeoffs in running speed versus compression ratio and , as a result produces different
compressed forms. Applying hash and signature after compression would result in invalid
authentication.
The Message encryption is applied after compression:
to strengthen cryptographic security. Because the compressed message has less
redundancy than original plaintext. Cryptanalysis is difficult.
Email Compatibility
The data generated via encryption, signature, hash and Zip functions consists of a stream
of arbitrary 8-bit octets.

However, many email systems only permit the use of blocks consisting of ASCII text (7-bit data).
To overcome the situation PGP provides the service of converting the raw 8-bit binary stream to a
stream of printable ASCII characters.

The Scheme used for this purpose is radix-64 conversion. Each group of 3 octets of binary data
is mapped into 4 ASCII characters. The use of radix-64 expands a message by 33%.

This scheme is very much useful in achieving certain level of confidentiality when a message is
signed but not encrypted.

The output will be unreadable to the third party. As an option, PGP can be configured to convert
to radix-64 format only the signature portion of the signed plaintext messages.
Segmentation and Reassembly
Email systems has certain restrictions related to maximum email message length to be
transmitted.

For example, some of the facilities impose a maximum length of 50,000 octets. Any message
longer than that must be broken up into smaller segments each of which is mailed separately.

To accommodate this restriction, PGP automatically subdivides a message that is too

large into segments that are small enough to send via e-mail.
The segmentation is done after all of the other processing, including radix-64
conversion.

Thus, the session key component and signature component appear only once, at the
beginning of the first segment.

At the receiving end, PGP must strip off all email headers and reassemble the entire
original block.
General Format of PGP Message
The Message format of PGP includes 3 components:
 Message
 Signature
 Session key component
Message includes actual data to be stored or
transmitted as well as a file name that includes
control information generated by PGP and a
timestamp that corresponds to the time of creation.

Signature includes Message digest, leading two

octets of MD acts as the check bits to
check the integrity of MD, Key ID of senders public
key(KUa) used to decrypt the message digest.
The message component and signature component
may be compressed using ZIP and may be
encrypted using a session key.
General Format of PGP Message contd…
The session key component
includes the session key (Ks)
and identifiers of recipients
public key (KUb) that was used
by the sender to encrypt the
session key. The entire block is
usually encoded with radix-64
encoding.
Radix-64 example: Suppose that out non-ASCII data that
needs to be converted into
ASCII using Base-64 encoding is :
10011011 10100010 11101001.
using 8-bit ASCII i.e. 155, 162, 233 in decimal.
After conversion to 6-bit ASCII the sequence is
100110-38 in decimal, .m. as per radix-64
11101058 in decimal, .6. as per radix -64
00101111 in decimal .L. as per radix -64
10100141 in decimal .p. as per radix-64
Hence, our original non-ASCII text of 155, 162 and 233 in
decimal
would be sent as .m6Lp.
S/MIME
S/MIME was developed by RSA to prevent forgery and interception of electronic
messages. S/MIME was created on the existing MIME protocol standard and it can be
integrated easily into the existing email and messaging products.

S/MIME is very similar to PGP. Both offer the ability to sign and/or encrypt messages.

S/MIME provides cryptographic security services for email applications such as:
 message integrity,
 authentication, and
 non-repudiation of origin,
 privacy, and data security.

The functionality of S/MIME is built into the vast majority of recent editions of email
software including Gmail, Outlook Express, Apple Mail, Mozilla Thunderbird, Lotus
Notes, Netscape Communicator, Gnus, KMail, Balsa, and Sun Java Messaging.
MIME Header: MIME version : This contains MIME version
MIME defines 5 headers that can be added number i.e. 1.1
to the original e-mail header: Content-Type: Describes the data contained in
1. MIME version the body of the message. The details provided
2. Content-type are sufficient so that the receiver email system
3. Content-Transfer encoding can deal with the received email message in an
4. Content-ID appropriate manner.
5. Content description The contents are specified as : Type / subtypes.
MIME specifies 7 content type and 15 content
subtypes.
Content-Transfer Encoding: Specifies the type of transformations that has been used to
represent the body of the message.
7-bit ASCII characters and short lines.
8-bitNon-ASCII Characters short lines
Binary  Non-ASCII characters with unlimited-length lines.
Base-64  6-bit blocks of data encoded into 8-bit ASCII characters
Quoted-printable Non-ASCII characters encoded as an equal to sign. Followed by an
MIME Header Contd…

Content-ID: Identifies the whole

message in a multiple-message
environment.
Content descriptor: Defines
whether body is image, audio or
video.
 Security at the Transport Layer: SSL and TLS
A general-purpose solution is to implement security just above TCP. The
foremost example of this approach is the Secure Sockets Layer (SSL) and the
follow on Internet standard of SSL known as Transport Layer Security (TLS).
At this level, there are two implementation choices. For full generality, SSL (or
TLS) could be provided as part of the underlying protocol suite and therefore be
transparent to applications.
Secure Socket Layer (SSL) / SSL Architecture
SSL 3.0 was originated by Netscape. SSL is designed to make use of TCP to
provide a reliable end-to-end secure service. SSL is not a single protocol but
rather two layers of protocols.

The SSL Record Protocol provides basic security services to various higher-
layer protocols.

In particular, the hypertext transfer protocol (HTTP), which provides the transfer
service for Web client/server interaction, can operate on top of SSL.

Three higher-layer protocols are defined as part of SSL: the Handshake

Protocol, the Change Cipher Spec Protocol, and the Alert Protocol. These SSL-
specific protocols are used in the management of
SSL exchanges.
Two important SSL concepts are the SSL
session and the SSL connection, which are
defined in the specification as follows:

Connection: A connection is a transport

(in the OSI layering model definition) that
provides a suitable type of service. For
SSL, such connections are peer-to peer
relationships.
The connections are transient. Every
connection is associated with one session.

Session: An SSL session is an association between a client and a server. Sessions are
created by the Handshake Protocol. Sessions define a set of cryptographic security
parameters, which can be shared among multiple connections. Sessions are used to
avoid the expensive negotiation of new security parameters for each connection.
SSL Record Protocol
The SSL Record Protocol
provides two services for SSL
connections:
Confidentiality: The
Handshake Protocol defines a
shared secret key that is used
for
conventional encryption of SSL
payloads.
Message Integrity: The
Handshake Protocol also
defines a shared secret key that
is
used to form a message
authentication code (MAC)
 SSL Record Protocol Contdd…
Figure indicates the overall operation of the
SSL Record Protocol. The Record Protocol
takes an application message to be
transmitted, fragments the data into
manageable blocks, optionally compresses
the data, applies a MAC, encrypts, adds a
header, and transmits the resulting unit in a
TCP segment. Received data are
decrypted, verified, decompressed, and
reassembled and then delivered to higher-
level users.
Change Cipher Spec Protocol
The Change Cipher Spec Protocol is
one of the three SSL-specific protocols
that use the SSL
Record Protocol, and it is the simplest.
This protocol consists of a single
message, which consists of a single
byte with the value 1. The sole purpose
of this message is to cause the
pending state to be copied into the
current state, which updates the cipher
suite to be used on this connection.
Alert Protocol
The Alert Protocol is used to convey SSL-
related alerts to the peer entity. As with
other applications that use SSL, alert
messages are compressed and encrypted,
as specified by the current state.
Each message in this protocol consists of
two bytes. The first byte takes the value
warning (1) or fatal (2) to convey the
severity of the message. If the level is fatal,
SSL immediately terminates the
connection. Other connections on the same
session may continue, but no new
connections on this session may be
established.
Handshake Protocol
The most complex part of SSL is the
Handshake Protocol. This protocol
allows the server and client to
authenticate each other and to
negotiate an encryption and MAC
algorithm and cryptographic keys to
be used to protect data sent in an
SSL record.
The handshake protocol
is used before any application data is
transmitted.
The Handshake Protocol consists of
a series of messages exchanged by
client and server.
SSL Handshake Protocol Message Types

The Handshake Protocol consists of

a series of messages exchanged by
client and server. All
of these have the format shown in
the following Figure. Each message
has three fields:
Type (1 byte): Indicates one of
10 messages..
Length (3 bytes): The length
of the message in bytes.
Content ($ 1 byte): The
parameters associated with this
message;
Transport Layer Security (TLS)
TLS is an IETF standardization initiative whose goal is to produce an Internet
standard version of SSL. The current draft version of TLS is very similar to
SSLv3. This section highlights the differences.

Version Number
The TLS Record Format is the same as that of the SSL Record Format
Message Authentication Code
There are two differences between the SSLv3 and TLS MAC schemes: the actual algorithm
and the scope of the MAC calculation. TLS makes use of the HMAC algorithm defined in
RFC 2104.
Pseudorandom Function
TLS makes use of a pseudorandom function referred to as PRF to expand secrets into blocks
of data for purposes of key generation or validation. The objective is to make use of a
relatively small shared secret key.
To make PRF as secure as possible, it uses two hash algorithms in a way that should
guarantee its security if either algorithm remains secure.
Alert Codes
TLS supports all of the alert codes defined in SSLv3 with the exception of
no_certificate. A number of additional codes are defined in TLS and a few are
here:
decryption_failed: A ciphertext decrypted in an invalid way; either it was
not an even multiple of the block length or its padding values, when checked,
were incorrect.
record_overflow: A TLS record was received with a payload (ciphertext) whose
length exceeds 214 + 2048 bytes, or the ciphertext decrypted to a length of greater
than 214 1 1024 bytes.
unknown_ca: A valid certificate chain or partial chain was received, but the
certificate was not accepted because the CA certificate could not be located or could
not be matched with a known, trusted CA.
access_denied: A valid certificate was received, but when access control was
applied, the sender decided not to proceed with the negotiation.
IP Security
Need for IP Security

There are systems that provide security at application layer (PGP, S/MIME)

Application layer security refers to methods of protecting web applications at the

application layer from malicious attacks that may expose private information.
Security is applied to the application layer specifically to protect against unauthorized
access and attacks.

Systems that provide security at transport layer (SSL,TLS,SSH), for establishing a secure
connection between a client and a server.
TLS (Transport Layer Security) is capable of authenticating both the client and the
server and creating a encrypted connection between the two.
 Need for IP Security Contd…

However the security at above two layers are may not be enough in some cases:

 Not all client/server programs are protected at the application layer Eg. PGP and S/MIME
protect only email application.

 Not all client/server programs at the application layer use the services of TCP (Transport
layer) to be protected by SSL or TLS. Some programs use the service of UDP.

 Many applications, such as routing protocols, directly use the service of IP, and the most
serious types of attacks included IP Spoofing, in which intruders create packets with
false IP addresses and exploit applications that use authentication based on IP address,
and various forms of eavesdropping and packet sniffing, in which attackers read
transmitted information, including logon information and database contents.

For all the above reasons security services are needed at the IP layer
IP Security Overview
IPsec is an extension to the IP protocol
which provides security to the IP and the
upper layer protocols.
It was first developed for the new IPv6
standard and then “backported” to IPv4.
The IPsec architecture is described in the
RFC2401.
In order to ensure the security of Internet,
Internet Architecture Board (IAB), in 1994
issued a report stating the issues and
mechanisms to secure the network
infrastructure from unauthorized
monitoring and control of network
traffic and the need to secure end user-
to-end-user traffic using authentication
and encryption.
 IP Header: General IP Header
IPSec Header: Also called authentication header, provides integrity protection,
authentication.
Secure IP Payload: Called Encapsulating Security Payload(ESP), provides optional
integrity protection and optional encryption.

Applications of IPsec

IP security provides the capability to secure communication across a LAN, across private
and public WAN‟s and Internet.
1. A company can build a secure virtual private network over the Internet or over a
public WAN. Advantage: - cost reduction and minimizing overhead.

Company main office  company Branch office

Applications of Ipsec Contd…
2. Secure remote access over the Internet, i.e. an end user can make local call to an ISP
and gain secure access to a company network.

Local call  ISP  Company network

3. Establishing secure communication with other organization, i.e. intranet and

extranet connectivity by ensuring authentication and confidentiality and providing a
key exchange.

One Company  Other company

4. Enhancing electronic commerce security.

IP security Architecture
IPSec architecture is a group of documents
that define IP security. The following
diagram shows the complete picture:
ArchitectureDefine General concepts,
security requirements, and general issues
and Mechanisms of IPSec.
ESP (Encapsulating Security payload)
The ESP protocol can both ensure the
integrity of the packet using a HMAC and
the confidentiality using encryption. The
ESP header is generated and added to the
packet.
AH (Authentication Header)  provides
support for the data integrity and
authentication for IP Packets.
IP security Architecture Contd….

Encryption algorithm This defines various encryption algorithms used for ESP.

Authentication Algorithm This defines how various authentication algorithms are used
for AH and for the authentication option of ESP.

DOI (Domain of Interpretation)Contains values needed for the other documents to

relate to each other i.e. identifiers for encryption, authentication algorithm and
operational parameter such as key lifetime.

Key Management Documents that describe determination and distribution of secret

keys.
IPSec Services
IPSec provides security services at the IP layer are:

Access control: enabling or disabling the access to resources.

Connectionless connectivity: Since IP is a connectionless protocol it provides Connectionless
connection i.e. datagram transmission and reception.
Data origin Authentication: Use data origin authentication, which enables the recipient to
verify that the packets have not been tampered with in transit (data integrity) and that they
originate from the expected sender (authenticity).
Rejection of replayed packets: Packet replay refers to capturing and then resending
packets on a network, with or without packet modification and those packets are
eliminated.
Confidentiality: Encryption
Limited traffic flow confidentiality: It is a mechanism which hides the substance of the
packet actual data (by adding padding bits)
Security Associations
 Secure Association (SA) is a cryptographically protected connection and it acts as a
set of rules.
 A security association (SA) is the establishment of shared security information
between two network entities to support secure communication.
 Used to manage authentication and confidentiality services between sender and
the receiver.
 A unidirectional or one-way relationship affords security services to the traffic
carried on it.
 For two-way secure exchange two SA‟s are required.
 Security Services afforded to an SA for use of AH or ESP, but not both.
 IPSec header contains SPI (Security Parameter Index) that identifies the SA‟s or
allows partner to look up the necessary data such as the key in SA database
 Two types of Security Associations exist:
 Transport Mode
 Tunnel mode

In transport mode, the original IP header is followed by the AH or

ESP header. If ESP is used in transport mode, only the upper-layer
(e.g., TCP, UDP, and IGMP) is encrypted. The IP header is not
encrypted. Typically, transport mode is used for end-to-end
communication between two hosts.
In tunnel mode, the original IP datagram, including the original IP
header, is enclosed, or encapsulated within a second IP datagram.

If ESP is used in tunnel mode, the original IP datagram, including

the original header, is encrypted.

If ESP is used in tunnel mode on gateways, the outer, unencrypted

IP header will contain the IP addresses of the gateways, and the
inner, encrypted IP header will contain the ultimate IP source and
destination addresses.
This prevents eavesdroppers from analyzing the network traffic
between the ultimate source and destination addresses.
Encapsulating Security Payload (ESP)
The ESP protocol can both ensure the integrity of the packet using a
HMAC and the confidentiality using encryption. After encrypting the
packet and calculating the HMAC the ESP header is generated and added
to the packet. Figure shows the format of an ESP packet. It contains the
following fields:
Encapsulating Security Payload (ESP) Contd…

Security Parameters Index (32bits): Identifies a SA to use for the decapsulation of the
ESP packet. The receiver uses this value to determine the security association with
which this packet should be identified. This works like port numbers in TCP and UDP
connections.
Sequence Number (32 bits): A monotonically increasing counter value. This sequence
number is used to protect against replay attacks. The receiver checks this field to
verify that a packet for a security association with this number has not been received
already. If one has been received, the packet is rejected.
Payload Data (variable): A transport-level segment (transport mode) or IP packet
(tunnel mode) that is protected by encryption.

The ESP trailer contains the following fields:

Padding (0-255 bytes): Extra bytes that may be required if the encryption algorithm
requires the plaintext to be a multiple of some number of octets.
Pad Length (8 bits): Indicates the number of pad bytes immediately preceding this
field. This field is used by the receiver to discard the Padding field.
Encapsulating Security Payload (ESP) Contd…

Next Header (8 bits): Identifies the type of data contained in the payload data field by
identifying the first header in that payload. Identifies the nature of the payload, such
as TCP or UDP.
Authentication Data (variable): A variable-length field (must be an integral number of
32-bit words) that contains the integrity check value computed over the ESP packet
minus the Authentication Data field. The length of the field is specified by the
authentication function selected. This field is optional, and is included only if the
authentication service has been selected for the SA in question.

ESP provides protection for upper layer protocols. The Signed area indicates where the
packet has been signed for integrity. The Encrypted area indicates what information is
protected with confidentiality.
Authentication header
The Authentication Header (AH) protocol provides data origin authentication, data
integrity, and replay protection. However, AH does not provide data confidentiality, which
means that all of your data is sent in the clear.
The Internet Engineering Task Force (IETF) formally defines AH in Request for Comment
(RFC) 2402, IP Authentication Header.
The following AH packet diagram shows how an AH packet is constructed and interpreted
Next header
The Next Header is an 8-bit field that identifies the type of the next payload
after the Authentication Header..
Payload length
Size of AH packet.
RESERVED
Reserved for future use (all zero until then).
Security parameters index (SPI)
Identifies the security parameters, which, in combination with the IP address,
then identify the security association implemented with this packet.
Sequence number
A monotonically increasing number, used to prevent replay attacks.
Authentication data
Contains the integrity check value (ICV) necessary to authenticate the packet;
it may contain padding.
IP Security Policy
An IPSec policy is nothing more than a set of rules that govern when and how Windows uses
the IPSec protocol. The IPSec policy interacts directly with the IPSec driver. The policy tells
Windows such things as which data to secure IPSec policies work by determining which IP
traffic should be secured and which IP packets should be left alone and which security
method to use. This is accomplished through the use of an IP filter list, individual IP filters,
and filter actions.
Then, provide the IPSec policies with some information about your network. This
information may include things like the security method to use, the connection type, and the
tunnel settings.
The security method simply dictates which security algorithms should be used during the
authentication process and which algorithms should be used for key exchanges. The
connection type refers to whether the policy should be applied to remote access
connections,
LAN connections, or all network connections regardless of the type. The tunnel settings are
only used if you‟re using IPSec over a virtual private network. The tunnel settings define the
DNS name or the IP address of the tunnel’s end point.
Internet Key Exchange (IKE)
The IPSec ESP and AH protocols provide integrity and authentication of IP
packets, but they are not the complete package.
IETF includes a protocol that provides several services, including:
 Negotiating which protocols, algorithms, and keys will be used in a
communication
 verifying the identity of the other party; and
 Managing and exchanging keys.
The key management portion of IPSec involves the determination and
distribution of secret keys.
The IPSec Architecture document mandates support for two types of key
management:
Manual : A system administrator manually configures each system with its
own keys and with the keys of other communicating systems. This is
practical for small, relatively static environments.
Automated : An automated system enables the on-demand creation of keys
for SA’s and facilitates the use of keys in a large distributed system with an
evolving configuration. An automated system is the most flexible but
requires more effort to configure and requires more software, so smaller
installations are likely to opt for manual key management.
The Two protocols of IPSec, the Internet Security Association and key
Management protocol (ISAKMP) / Oakley key exchange protocol
automatically handles exchange of secret symmetric keys between sender and
receiver.
ISAKMP:
- a protocol to establish a framework of authentication and key
exchange.- is based on Diffie- Hellman model of key generation, in which the
two parties share information beforehand to ensure the identity of the other
party.
- Defines procedures and provides a framework for Internet key
management packet formats to establish, negotiate, modify and delete
Security Associations.
by itself does not dictate a specific key exchange algorithm; rather,
ISAKMP consists of a set of message types that enable the use of a
variety of key exchange algorithms.
Oakley:
- which describes a series of key exchange defining in detail the services
provided by them
- The Oakley Key Determination Protocol is a key-agreement protocol
that allows authenticated parties to exchange keying material across an
insecure connection using the Diffie-Hellman key exchange algorithm.

Cryptographic Suites for IPSec

Cryptographic algorithms defined for use with IPsec include:
• HMAC-SHA1 for integrity protection and authenticity.
• Triple DES-CBC for confidentiality
• AES-CBC for confidentiality.