What is an IPsec based VPN?

by | 1 Nov 2017

IPsec Networks

IPsec is a protocol suite for securing IP (transport layer) communications between peers by authenticating and encrypting each packet of communication.

In the network diagrams, the two red lines represent IPsec tunnels from a VNS3 Controller to the two remote firewall devices. The London and Chicago locations are two different endpoints.

VNS3 overlay network

An IPsec tunnel is built between two endpoint devices that “speak” the network standard. You can think of the following two phases and these two devices cordially introducing themselves and establishing a connection via an appropriately firm and friendly handshake.

IPsec negotiation is divided into two phases:

  1. Introductions: Peers find each other over the internet and trade security parameters
  2. Conversation: Peers exchange information using agreed-upon security standards

What is an IPsec based VPN?

An IPsec based VPN is made up of two parts:

  1. Internet Key Exchange protocol (IKE), underlying port UDP 500
  2. IPsec protocol (ESP), underlying Protocol 50 or if using “nat-traversal” UDP 4500

Basically there is an initial brief interaction where one or each of the devices attempt to discover each other, via the Internet, they then trade Phase 1 (IKE) parameters and attempt to get a Phase 1 (sometimes called IKE or ISAKMP) connection which creates the keys used to encrypt Phase2. They then trade Phase 2 parameters and attempt to create an encrypted Phase 2 (sometimes called IPSec SA or ESP) tunnel connection.

IPsec negotiation is divided into two phases:

Phase 1: Initial negotiation phase. Peers find each other over the internet and trade security parameters in order to create “session keys” that prove they are who they claim to be. This is what security experts call a key exchange, and the peers use the Internet Key Exchange Protocol (IKE) to validate each other’s security claims.

Phase 2: Peers trade Phase 2 security parameters. The peers trust each other, and now can create an encrypted tunnel that connects them using IPsec Protocol Encapsulating Security Payload (ESP) to encrypt the IP Packet. That way, no one else can see what data is traveling between the peers. Even if someone was able to see the data, it is all encrypted and unreadable without a secure key.

How does an IPsec VPN work?

IPsec is able to secure packets as they travel across an IP network, usually between two sites separated by the public Internet. IPsec uses cryptographic security services to authenticate and encrypt each data packet as it travels two endpoints and protects that data from traffic sniffers.

IPsec connections ensure data isn’t read or tampered with when it travels across any type of network. IPsec is especially important when you’re building hybrid deployments between public cloud providers or the public internet.

IPsec has 2 types of implementation: a host-to-host transport mode, or network tunneling mode:

Transport mode
In transport mode, only the payload of the IP packet is usually encrypted and/or authenticated. Transport mode does not change the routing, since IP headers aren’t modified or encrypted. Transport mode works best in host-to-host connections.

Tunnel mode
In tunnel mode, the entire IP packet is encrypted and/or authenticated. Tunnel mode must encapsulated the headers into a new IP packet with a new IP header. VPNs use tunnel mode for network-to-network communications, host-to-network communications, and host-to-host communications. Tunnel mode does support NAT traversal.