Advanced IPSec
Module 15
© 2013 Fortinet Inc. All rights reserved.
The information contained herein is subject to change without notice. No part of this publication including text, examples, diagrams
or illustrations may be reproduced, transmitted, or translated in any form or by any means, electronic, mechanical, manual, optical
or otherwise, for any purpose, without prior written permission of Fortinet Inc.
• In the 201 course, the IPSec module covered:
» IPSec protocol basics
» Interface-based VPN
» Policy-based VPN
» Overlapping subnets
» Site-to-site deployment
» VPN configuration
» Log messages
Module Objectives
• By the end of this module participants will be able to:
» Configure dialup VPN access
» Implement advanced topologies
» Debug IPSec communications
IPSec Review
• Suite of protocols for securing IP communications by authenticating
and/or encrypting packets:
» Internet Key Exchange (IKE)
» Encapsulation Security Payload (ESP)
• IP protocol number 50
• Provides data integrity and encryption
» Authentication Header (AH)
• IP protocol number 51
• Only provides data integrity
• Not used by the FortiGate unit
Either ESP or AH can be
used to transport user traffic
IKE Review
• UDP port 500 (and UDP port 4500 when NAT-T is used)
• Based on the Internet Security Association and Key Management
Protocol (ISAKMP)
• Protocol for the establishment of Security Associations (SAs)
• A Security Association (SA) is a bundle of algorithms and parameters
for processing the secured packets from one node to another:
» One IPSec SA is required per each traffic direction
» So, if there are 4 IPSec tunnels, there are 8 IPSec SAs
• IKE phases:
» One phase 1 per VPN tunnel
» One or more phase 2s per phase 1
• The Diffie-Hellman protocol is a ‘key-agreement’ protocol to allow a
pair of peers to communicate over an unsecure channel and
independently calculate a shared secret key using only public keys
• The shared secret key is then used to calculate keys for symmetric
encryption algorithms (such as 3DES, AES) and symmetric
authentication (HMACs)
• With Perfect Forward Secrecy (PFS) a new common secret key is
recalculated each time the phase 2 session key expires
IKE/ISAKMP Phase 1 Review
• Perform authentication and Diffie-Hellman exchange to generate a
common secret session key
• Two keys are then derived from the session key:
• One to encrypt phase 2 messages
• One to authenticate phase 2 messages
• Two possible modes:
• Main mode: 6 packets
• Aggressive mode: 3 packets
• Twp types of authentication:
• Pre-shared key
• X.509 Certificates
Phase 1 Main Mode with Pre-Shared Key
Initiator ISAKMP Policy
Responder ISAKMP Policy
Initiator Diffie-Hellman key
Responder Diffie-Hellman key
Initiator ID and hash payloads
Responder ID and hash payloads
• As the first packet coming from the Initiator does not include its peer ID,
the responder can only identify the peer by its IP address
• Because the peer IP address for dial-up VPNs might change, there
cannot be more than one dial-up VPN in the responder configuration
Phase 1 Aggressive Mode with Pre-Shared Key
Initiator ISAKMP Policy, DiffieHellman key and ID
Responder ISAKMP Policy, DiffieHellman key and ID and hash payloads
Initiator Hash Payload
• Peer can be identified using a wide range of identifiers, not only the
source IP address, but also the peer ID
• So, there can be more than one dial-up VPN
• The responder will use the peer ID included in the first packet to
identify the peer and apply the right VPN configuration
Quick Mode Selectors
• Are used to identify and direct traffic to the appropriate phase 2 in
cases where multiple phase 2s exist
• Allow for SAs with different granularities
• Similar to firewall policies:
» VPN traffic that does not match the selector is dropped
• Selectors support:
» Destination and source IP addresses
» Protocol number, and source and destination ports
• In point-to-point VPNs, the selectors configuration at both ends must
mirror each other:
» The source at one end must be the destination at the other end
NAT Transversal (NAT-T)
• ESP might have problems in some NAT environments
» So, enabling NAT-T in the phase 1 configuration is recommended in those cases
• NAT-T detects if there are any devices along the transmission path
doing NAT
• If that is the case, the phase 1 negotiation changes from using UDP
port 500 to UDP port 4500:
» All the subsequent packets (including phase 2s and encrypted user data) is
transmitted using UDP port 4500 instead of ESP
• If that is not the case, the phase 1 and phase 2s keep using UDP port
500 and the encrypted user data is sent using ESP
Extended Authentication (XAuth)
• Optional setting in the phase 1 configuration
• Enforce an additional level of authentication:
» Remote users or devices must sent their credentials (username and password)
before starting the phase 2 negotiation
• Additional user authentication is exchanged after the IKE phase 1 SA
is established:
» That is why XAuth is sometimes called IPSec phase 1.5
Type of Remote Gateways
• Static IP Address (point-to-point):
» The IP address of the remote gateway does not change
• Dynamic DNS (point-to-point):
» The IP address of the remote gateway might change but it has a dynamic DNS
• Dialup (point-to-multipoint)
» The remote peer does not have a dynamic DNS name and its IP address might
VPN Topologies
• Point-to-point (covered in the 201 course)
• Dial-Up (point-to-multipoint)
• Hub-and-Spoke *
• Full Meshed *
• Partial Meshed *
* These 3 topologies are built combining or using point-to-point and/or dial-up VPNs
Dialup VPN
Mobile User
Branch office
Branch office
Branch office
Branch office
Full and Partial Meshed
Full Mesh
Partial Mesh
VPN Topology Comparison
• Hub-and-Spoke
» Few VPN tunnels (low resources)
» Easy to maintain
» Bandwidth requirements at central hub
» Single point of failure
• Full/Partial Mesh
» More VPN tunnels (more resources)
» More difficult to maintain
» Accurate map required
Dial-Up VPN Configuration Steps
• In each site, create:
» The phase 1
» At least one phase 2
» The firewall policies
» Static or dynamic routing might be required
• Remember from the 201 course:
» Additional routing configuration usually is not required when working with policybased VPNs
» Interface-based VPNs require two firewall policies, one for each traffic direction
» Policy-based VPNs require only one firewall policy (it is applied bi-directionally)
Central Site Phase 1 Configuration
Remote Gateway must be
Dialup User
Peer/Local ID and aggressive
mode must be used for more
than one dial-up VPN
We can optionally enable
XAuth Server
NAT-T should be enabled for
mobile users
Central Site Phase 2 Configuration
Quick Mode Selectors are
Remote Site Phase 1 Configuration
Remote Gateway must be
either Static IP or Dynamic
Peer/Local ID and aggressive
mode must be used for more
than one dial-up VPN
We can optionally enable
client XAuth
Remote Site Phase 2 Configuration
A static route to this network
will be automatically added in
the central site
Central site local
IKE Mode Config
• A method to automatically configure the IP settings of the VPN dial-up
clients, only supported in interface-mode VPN
• FortiClient VPNs use Mode Config by default to get the IP settings
from the FortiGate unit
FortiClient VPN Configuration Wizard
• Simplifies the creation of VPNs for FortiClient mobile users
• It creates a dial-up phase 1 with the following settings:
» Interface-based
» XAuth enabled
» IPSec Mode-Config
» Aggressive mode
• It creates a phase 2
• It does not add the required firewall policies and routing
Redundant VPNs
• Only fully supported by interface-based VPNs
• If the primary VPN connection fails, the FortiGate units re-route the
traffic through the backup VPN
• Partially redundant: Only one peer has redundant connections
Branch office
• Fully Redundant: Both peers have redundant connections
Branch office
Redundant VPN General Configuration Steps
1. One phase 1 configuration for each path between the two peers. Dead
Peer Detection (DPD) must be enabled in both ends
2. At least one phase 2 definition for each phase 1 configuration
3. One static route for each path, with different distance values to
prioritize the routes. Dynamic routing can be used instead
4. Two firewall policies per IPSec interface, one for each traffic direction
Primary VPN
Branch office
Backup VPN
VPN Troubleshooting Tips
• Always debug on the responder side
• Most connection failures are due to configuration mismatches, so
» Mode setting for ID protection (main or aggressive)
» Authentication methods
» Pre-shared keys
» Client must have at least one set of matching Phase 1 and Phase 2 settings as
configured on FortiGate unit
• Ping and trace to the remote network or client to verify that the
connection is up
VPN Troubleshooting
• For IPSec real-time debugging:
diagnose debug reset
diagnose vpn ike log-filter ?
diagnose debug application ike 255
diagnose debug enable
• The output is long and shows details about what is happening with the
phase 1 and phase 2 negotiations
Real-Time Debug During the Phase 1 (Main Mode)
• These are the most important messages displayed by the real-time
debug during a successful Main Mode negotiation:
ike 0: comes>,ifindex=2....
ike 0: IKEv1 exchange=Identity Protection id=4497f0b077c742b5/0000000000000000 len=296
ike 0:4497f0b077c742b5/0000000000000000:8: responder: main mode get 1st message...
ike 0:4497f0b077c742b5/0000000000000000:8: SA proposal chosen, matched gateway Remote
ike 0: found Remote 2 ->
ike 0:Remote:8: sent IKE msg (ident_r1send):>, len=160
ike 0: comes>,ifindex=2....
ike 0:Remote:8: responder:main mode get 2nd message...
ike 0:Remote:8: sent IKE msg (ident_r2send):>, len=292
ike 0:Remote:8: ISAKMP SA 4497f0b077c742b5/fbbb59b259a0fc3e key 24:DCD18FBE7CFA138E27B06F
ike 0: comes>,ifindex=2....
ike 0:Remote:8: responder: main mode get 3rd message...
ike 0:Remote:8: PSK authentication succeeded
ike 0:Remote:8: authentication OK
ike 0:Remote:8: established IKE SA 4497f0b077c742b5/fbbb59b259a0fc3e
Receive first Main
mode packet
Found a phase 1 that
Received second Main
mode packet
Received third Main mode
Pre-shared key
authentication OK
Real-Time Debug During the Phase 2
• These are the most important messages displayed by the real-time
debug during a successful phase 2 negotiation:
Ike 0:Remote:7:22: responder received first quick-mode message
ike 0:Remote:7:22: peer proposal is: peer:0:, me:0:
ike 0:Remote:7: sent IKE msg (quick_r1send):>, len=356
ike 0: comes>,ifindex=2....
ike 0:Remote:7:P2:22: replay protection enabled
ike 0:Remote:7:P2:22: SA life soft seconds=1750.
ike 0:Remote:7:P2:22: SA life hard seconds=1800.
ike 0:Remote:7:P2:22: IPsec SA selectors #src=1 #dst=1
ike 0:Remote:7:P2:22: src 0 7 0:
ike 0:Remote:7:P2:22: dst 0 7 0:
ike 0:Remote:7:P2:22: add IPsec SA: SPIs=6e13ca19/8f1ce9ae
ike 0:Remote:7:P2:22: added IPsec SA: SPIs=6e13ca19/8f1ce9ae
ike 0:Remote:7:P2:22: sending SNMP tunnel UP trap
Receive first Quick
mode packet
Quick mode selectors
proposed by the remote
Negotiated Quick mode
SA created and tunnel up
Debug Flow of IPSec Packets
FGT # diagnose debug flow filter addr
FGT # diagnose debug flow function enable
FGT # diagnose debug flow show console enable
FGT # diagnose debug flow trace start 10
FGT # diagnose debug enable
Valid IP address in the
remote site
id=36871 trace_id=1 msg="vd-root received a packet(proto=1,> from internal."
The packet is
id=36871 trace_id=1 msg="allocate a new session-0000004f"
id=36871 trace_id=1 msg="find a route: gw- via wan1"
id=36871 trace_id=1 msg="Allowed by Policy-1: encrypt"
… and sent through the
tunnel ‘HQ’
id=36871 trace_id=1 msg="enter IPsec tunnel-toHQ"
id=36871 trace_id=1 msg="encrypted, and send to with source"
id=36871 trace_id=1 msg="send to via intf-wan1"
• Lab 1: IPSec VPN
» Ex 1: Configuring IPSec VPNs
» Ex 2: Testing the IPSec VPN Configuration
» Ex 3: Configuring Semi-Redundant IPSec VPNs
» Ex 4: Testing the Semi-Redundant IPSec VPN Configuration
» Ex 5: Configuring OSPF
» Ex 6: Testing the OSPF Configuration
» Ex 7: Enabling Bi-Directional Forwarding Detection
• Lab 2: IPSec VPN with FortiClient
» Ex 1: Configuring the FortiGate as a VPN Gateway
» Ex 2: Configuring FortiClient Connect
» Ex 3: Testing the FortiGate to FortiClient IPSec VPN Connection
Classroom Lab Topology