Strengthening the different layers of IT networks

Getting started on designing a network is a task of formidable proportions, but all the more so if you are looking to build a secure IT network.

Often, IT system administrators are not fully familiar with the security mechanisms that can be implemented at each layer of abstraction. So, let’s go through the security options available at each level of the network, which will enable you to establish at which level a security breach occurred, and what actions can be taken to fix it.

A little background

The Open Systems Interconnection (OSI) model was developed in 1983 by the International Organization for Standardization (ISO). Ever since then, it has been the go-to model for understanding and analyzing how networks function. Its influence has been so great that the majority of communications protocols in use today are structured around it.

Nonetheless, the OSI model was born from a theoretical perspective, and despite its precision in encapsulating how a network functions, it ended up relinquishing the practical ground to new designs built on top of it, such as TCP/IP—the most commonly used model today. More than a model, it is a stack or suite of protocols used simultaneously, which allows the network to function.

As well as helping in the development of telecommunications protocols, this can be used as a basis for understanding how we can apply security strategies right across the length and breadth of an IT network’s structure. In the same way that the failure of one of these layers to provide its respective services results in the layers above it not being able to successfully fulfill its objectives, a vulnerability that compromises any layer renders useless any other protection measure that has been implemented in the layers of abstraction above it.

We can make a comparison between network security layers and the layers of an onion: it’s a question of implementing measures to compensate for weak points throughout all the layers, making sure to correctly configure the protocols in the protocol stack used, on all the mechanisms involved in the network.

For purposes of analysis, we will put together a new reference model combining the characteristics of the OSI model and the TCP/IP stack, as shown below. Then, we’ll use it to study which aspects of security relate to each layer.

1) Physical Layer

This deals with sending bits via analog and digital signals through various means of communication. In this layer, we find protocols like Ethernet, PPP, and Frame Relay, among others.

From an IT security perspective, at this level we are concerned with preventing unauthorized third parties from getting into the system. Measures include implementing a warning and surveillance system and arranging barriers and mechanisms to control access to the system on which the IT network is deployed.

Another risk to keep in mind is the “internal threat,” in other words, authenticated staff carrying out unauthorized actions, or pursuing a harmful goal. The arrangement of cables must not allow eavesdropping, and to achieve this it is necessary to restrict access to the telecommunications room and protect the cabling and equipment so they cannot be deliberately damaged for the purpose of attacking the availability of the service.

2) Data Link Layer

This deals with addressing at the level of frames, acting as a link between the network layer (packets with IP addresses) and the physical layer (signals). Switches are the most important mechanism at this level, and as such, data link security focuses on the correct configuration of these devices. For example, we have to combine the blocking of physical access with deactivation at the logic level of unused ports, in order to prevent fraudulent connections which could lead to eavesdropping, flooding attacks, or ARP spoofing.

This layer also includes the correct choice of secure protocols for communication. For example, in the case of wireless, we should use the WPA2 or WPA protocol whenever possible, in preference to WEP.

Another important element of this instance is the correct configuration of virtual networks. A VLAN is a local area network that, although it has all the attributes of a network at the logic level, does not exist as such on a physical level, but rather is created through the individual indication of identifiers on frames flowing through the network.

VLANs play a critical role in the system’s security, contributing to segmentation of the network and separation of traffic, allowing it to be better organized and quickly analyzed.

3) Network Layer

This deals with the routing of packets between networks, connecting different broadcasting domains. The addressing model used at this level is that of IP addresses, and the mechanism that centralizes traffic management is the router.

Activating the various security characteristics included in this equipment is important for preventing unauthorized control of it. Use of strong passwords and correct configuration of management protocols through encrypted connections are some of the measures that can be taken to protect this equipment.

Additionally, we have to keep in mind vulnerabilities that could exist in routing protocols, such as RIP and OSPF, as they could lead to the injection of false routers. Understanding these protocols will facilitate secure management of the network.

The IPv4 protocol does not include security mechanisms to protect communications, so the network administrator must consider other additional measures for protecting data. For example, they can choose to encrypt all network packets, or exchange a session key to protect the connection.

For this reason, we should consider implementing IPSec: the suite of protocols responsible for providing security features to the IP protocol, serving for the use of Virtual Private Networks, or VPNs. Its optional use of IPv4 will become obligatory for IPv6. It supports two modes of implementation

• Transport mode, whereby the ends of the communication protect it by encrypting only the packet’s payload

• Tunnel mode,used for secure connection between networks, where each IP packet is encrypted and encapsulated within another

In addition, there is a series of other problems that affect this layer. In particular, the possibility of an attacker trying to send data from equipment with a certain IP address when in reality they are doing so from another address—a process called IP spoofing. One way to minimize such attacks is to include authentication processes in the application layer, together with data encryption mechanisms.

Likewise, activating firewalls on the network layer can reduce such attacks, as these devices can delineate the inside of the network from the outside, making its internal/external ports correspond with the assigned address space. That is, a firewall can detect a packet that claims to come from inside, but in fact came in through the external port, and can therefore reject it or generate the corresponding warning.

Another key component of IT security at the network layer is the Access Control List. This allows or refuses connections with equipment belonging to other networks, depending on the protocol, ports, or IP addresses involved in the communication.

Correctly configuring ACLs is a task of considerable magnitude, as it requires knowledge of the protocols that will be used on the network, and knowledge of the network’s design. Any incorrect configuration could lead to authorizing fraudulent traffic or refusing legitimate connections.

4) Transport Layer

This takes data from the application and splits it into segments which will then be sent to the network layer. The preferred protocols at this level are TCP and UDP, which are responsible for establishing a real link from origin to destination. As such, the addressing model is the port number.

The security concerns at this level relate to encryption of the data transferred, authentication of the parties involved, prevention of tampering with data integrity, and avoidance of replay attacks.

With regard to TCP, one of the best-known types of attacks is the SYN attack. This takes advantage of some of the weaknesses in the three-channel model for establishing a connection. It involves malicious equipment sending multiple SYN requests to another terminal, and then stopping responding. The destination host will await confirmation, keeping the attempted connection on hold. If the maximum number of threads of unconfirmed connections that the host can support is exceeded, the host will be rendered unable to accept new communications, resulting in a denial of service.

One solution to this problem is to increase the number of unconfirmed connections on the server, or to decrease the amount of time the server will wait for confirmation (75 seconds by default).

The use of layer 4 secure communication protocols, such as SSL, TLS or SSH, enables data to be protected by encrypting it, and should be considered when setting up connections for remote management of devices.

Finally, as the connection is end-to-end in the transport layer, increasing protection at the terminals is the first step for preventing fraudulent connections. Updating your applications in order to protect the network from any vulnerabilities and errors that might arise is the very first measure for reinforcing your equipment.

5) Application Layer

The application layer in the TCP/IP stack, and its equivalent layers in the OSI model, deals with session management and applications executed on the equipment.

Here, we find a varied combination of protocols that allow the terminals to access numerous services. These include SMTP, POP, IMAP, DNS, HTTP, HTTPS, DHCP, FTP, and TFTP. Configuration of these services is dependent on the administrator’s experience, and it is important to be careful to prevent bad configurations from returning an access port to the network.

Implementing a high-level firewall enables increased control of network traffic. Unlike firewalls at the network layer or transport layer, firewalls at the application layer allow packet filtering based on a wide range of options, including a vast array of protocols.

In addition, at this layer we find Intrusion Detection Systems (IDS), Intrusion Prevention Systems (IPS), and other complete security solutions. These applications function on the entire TCP/IP stack, being able to detect dangerous behavior according to various criteria, and issuing warnings as appropriate.

Another relevant aspect, once this layer of abstraction has been implemented, is user education, and the definition of security policies for the IT department. Ensuring the security of IT equipment is a case of pairing the correct implementation of the technical defense mechanisms with the right instructions to the users of the equipment, and the administrators who have power over them. We cannot consider any one of these alone.

Conclusion

As we can see, each layer of the network encapsulates a set of security protocols, equipment and techniques that can be implemented to make it difficult for attackers to achieve their goal. Knowledge of these concepts will enable us to carry out a better analysis of the level of protection on our network, and any vulnerabilities and breaches that might exist.

By combining these principles with security architectures (such as demilitarized networks, physical and logic segments, tarpits, or honeypots), by applying active defense techniques, and carrying out regular network pentesting processes, network administrators can increase their network’s security, correcting its vulnerabilities at its very core.