Common IPsec Issues

Systematic Approach

IPsec troubleshooting requires a methodical approach. Start with basic connectivity, then verify Phase 1, Phase 2, and finally data flow.

IPsec Troubleshooting Methodology

1. Basic Connectivity: Can peers reach each other on UDP 500/4500?
2. Phase 1 (IKE): Authentication, policy matching, DH exchange
3. Phase 2 (IPsec): Proxy-ID matching, ESP/AH negotiation
4. Data Flow: Routing, NAT, firewall policies

Top 10 IPsec Problems

# Issue Symptoms Common Causes Quick Fix
1 Phase 1 Fails No IKE SA established PSK mismatch, policy mismatch Verify PSK and crypto policies
2 Phase 2 Fails IKE up, no IPsec SA Proxy-ID mismatch, ACL issues Check interesting traffic
3 Asymmetric Routing One-way connectivity Missing return routes Add static routes
4 NAT Issues Frequent disconnects NAT-T not working Enable NAT-T, check ports
5 MTU Problems Large packets dropped Path MTU discovery fails Lower MTU, enable MSS clamping
6 Certificate Issues Authentication failures Expired/invalid certificates Renew certificates, check CRL
7 Firewall Blocking No initial connection UDP 500/4500 blocked Open firewall ports
8 Time Sync Issues Intermittent failures Clock skew > 5 minutes Synchronize time with NTP
9 DPD Failures Tunnels marked down DPD timeout too aggressive Adjust DPD timers
10 Rekeying Problems Periodic disconnects Lifetime mismatch Standardize SA lifetimes

Error Message Decoder

Common Error Messages and Meanings:

  • "MM_NO_STATE": IKE message received for non-existent SA
  • "INVALID_PAYLOAD_TYPE": Unsupported IKE payload received
  • "NO_PROPOSAL_CHOSEN": No matching Phase 1 or Phase 2 proposals
  • "AUTHENTICATION_FAILED": PSK or certificate authentication failed
  • "TS_UNACCEPTABLE": Traffic selector (proxy-ID) mismatch
Next: Debug Commands

Debug Commands

Debug Caution

Debug commands can generate significant CPU load and log messages. Use sparingly in production and always disable when finished.

Cisco ASA Debug Commands

# Enable logging to see debug output
logging enable
logging buffered debugging
logging console debugging

# IKEv2 Debug Commands
debug crypto ikev2 platform 255
debug crypto ikev2 protocol 255

# IPsec Debug Commands  
debug crypto ipsec 255

# Condition-based debugging (recommended for production)
debug crypto condition peer 203.0.113.1
debug crypto ikev2 protocol 255
debug crypto ipsec 255

# View active debug conditions
show debug crypto condition

# Clear specific debug condition
clear debug crypto condition peer 203.0.113.1

# Disable all debugging
undebug all
no debug all

Show Commands for IPsec Status

Command Purpose Key Information
show crypto ikev2 sa Display IKE Phase 1 SAs State, peer IP, lifetime, DPD status
show crypto ipsec sa Display IPsec Phase 2 SAs SPI, encaps/decaps counters, protocols
show crypto map Display crypto map configuration Peers, ACLs, proposals, interface binding
show crypto ikev2 stats IKE statistics and counters Success/failure rates, message counts
show crypto engine connections Hardware crypto acceleration Active flows, acceleration status
show vpn-sessiondb detail Remote access VPN sessions User info, IP assignment, duration

Juniper SRX Debug Commands

# Enable IKE debugging
set security ike traceoptions file ike-debug
set security ike traceoptions flag all

# Enable IPsec debugging
set security ipsec traceoptions file ipsec-debug  
set security ipsec traceoptions flag all

# View IKE security associations
show security ike security-associations
show security ike security-associations detail

# View IPsec security associations
show security ipsec security-associations
show security ipsec security-associations detail

# View IKE statistics
show security ike statistics

# Monitor real-time IKE events
monitor start ike-debug.log

# Clear security associations
clear security ike security-associations
clear security ipsec security-associations

# Disable debugging
delete security ike traceoptions
delete security ipsec traceoptions

Linux strongSwan Debug Commands

# View IPsec status
ipsec status
ipsec statusall

# Start connection
ipsec up tunnel-name

# Stop connection  
ipsec down tunnel-name

# Reload configuration
ipsec reload

# Live debugging
ipsec stroke loglevel ike 4
ipsec stroke loglevel cfg 4

# View logs
tail -f /var/log/charon.log

# Test connectivity
ipsec stroke status

# Show installed policies
ip xfrm policy
ip xfrm state

Debug Output Analysis

Key Debug Messages to Look For:

  • Phase 1 Success: "IKE Initiator: New Phase 1, Intf outside, IP 203.0.113.1"
  • Authentication: "IKE: Peer authenticated using pre-shared key"
  • Phase 2 Success: "IPsec: New inbound ESP SA (SPI= 0x12345678)"
  • Traffic Match: "IPsec: Flow_permit() ACL match"
  • Encryption: "IPsec: Encapsulating packet with ESP"
Previous: Common Issues Next: Packet Analysis

Packet Analysis

Deep Inspection

Packet capture and analysis provide definitive evidence of IPsec behavior, revealing protocol details invisible through logs alone.

Capture Strategies

Inside Interface

  • Unencrypted traffic
  • Original packet headers
  • Application-layer protocols
  • Before IPsec processing

Outside Interface

  • Encrypted traffic
  • ESP/AH headers
  • IKE negotiation messages
  • After IPsec processing

Cisco ASA Packet Capture

# Create capture on outside interface for IPsec traffic
access-list IPSEC-CAPTURE extended permit udp any any eq 500
access-list IPSEC-CAPTURE extended permit udp any any eq 4500  
access-list IPSEC-CAPTURE extended permit esp any any
capture OUTSIDE-CAP access-list IPSEC-CAPTURE interface outside

# Create capture on inside interface for unencrypted traffic
access-list INSIDE-CAPTURE extended permit ip 192.168.1.0 255.255.255.0 192.168.2.0 255.255.255.0
capture INSIDE-CAP access-list INSIDE-CAPTURE interface inside

# View capture buffer
show capture OUTSIDE-CAP
show capture INSIDE-CAP

# Export capture to PCAP file
copy /pcap capture:OUTSIDE-CAP tftp://192.168.1.100/outside.pcap

# Clear captures when done
no capture OUTSIDE-CAP  
no capture INSIDE-CAP

Wireshark Analysis Filters

# IKE traffic (Phase 1)
isakmp or udp.port == 500

# NAT-T traffic
udp.port == 4500

# ESP traffic  
ip.proto == 50

# AH traffic
ip.proto == 51

# Specific peer conversation
ip.addr == 203.0.113.1 and (isakmp or esp or ah)

# IKE informational messages
isakmp.flags.msgtype == 1

# IKE authentication failures
isakmp.notify_type == 24

# Large IKE packets (fragmentation)
udp.length > 1500 and udp.port == 500

Reading IKE Packet Details

IKE Header: Version, Exchange Type, Flags, Message ID, Length
SA Payload: Proposed encryption, authentication, DH group
Key Exchange: Diffie-Hellman public keys
Nonce: Random values for key generation
Identification: Peer identity (IP, FQDN, email)
Authentication: PSK hash or certificate signature

ESP Packet Analysis

Field Size Purpose Wireshark Display
SPI 4 bytes Security Parameter Index esp.spi
Sequence 4 bytes Anti-replay counter esp.sequence
Payload Variable Encrypted original packet esp.encrypted_data
Padding 0-255 bytes Block cipher alignment esp.pad
Auth Data Variable Integrity Check Value esp.auth

Packet Analysis Best Practices

Best Practices

  • Capture on both inside and outside interfaces
  • Use specific capture filters to reduce noise
  • Correlate timestamps between captures
  • Export to PCAP for detailed analysis
  • Clear captures when troubleshooting complete

Considerations

  • Captures consume device memory
  • High-volume captures impact performance
  • ESP payload is encrypted (unreadable)
  • IKE packets may be fragmented
  • NAT-T adds UDP header overhead
Previous: Debug Commands Next: Case Studies

Real-World Case Studies

Learning from Experience

These real-world scenarios demonstrate common IPsec problems and their solutions, providing practical troubleshooting experience.

Case Study 1: Site-to-Site Tunnel Partially Working

Problem Description:

A site-to-site IPsec tunnel between headquarters and branch office allows traffic from HQ to branch, but not from branch to HQ.

Symptoms:

  • IKE Phase 1 and Phase 2 both show as "UP"
  • HQ users can access branch servers (192.168.2.0/24)
  • Branch users cannot access HQ servers (192.168.1.0/24)
  • ESP encap counters increment only on HQ side

Investigation Steps:

# Check crypto maps on both sides
HQ-ASA# show crypto map
Branch-ASA# show crypto map

# Verify interesting traffic ACLs
HQ-ASA# show access-list CRYPTO-ACL
Branch-ASA# show access-list CRYPTO-ACL

# Check routing tables
HQ-ASA# show route 192.168.2.0
Branch-ASA# show route 192.168.1.0
Root Cause:

Branch office had asymmetric routing - return traffic to HQ was taking a different path that bypassed the IPsec tunnel.

Solution:

# Add static route on branch office
Branch-ASA(config)# route inside 192.168.1.0 255.255.255.0 192.168.2.1

# Alternative: Use route-based VPN with tunnel interfaces
Branch-ASA(config)# interface tunnel 1
Branch-ASA(config-if)# tunnel source 10.2.2.1  
Branch-ASA(config-if)# tunnel destination 10.1.1.1
Branch-ASA(config-if)# ip address 172.16.1.2 255.255.255.252
Branch-ASA(config)# route outside 192.168.1.0 255.255.255.0 tunnel 1

Case Study 2: Remote Access VPN Performance Issues

Problem Description:

AnyConnect users experience very slow performance, especially with large file transfers. Small web browsing works normally.

Symptoms:

  • TCP connections work but are extremely slow
  • Ping response times are normal (low latency)
  • Large downloads fail or timeout
  • Web browsing of small pages works fine

Investigation Commands:

# Check interface MTU values
show interface outside
show interface inside

# Look for fragmentation in interface statistics  
show interface outside | include fragments
show interface inside | include fragments

# Check NAT-T status
show crypto ikev2 sa detail

# Verify MSS clamping configuration
show running-config sysopt
Root Cause:

Path MTU Discovery was failing due to ICMP filtering, causing large packets to be fragmented or dropped. The NAT-T UDP encapsulation added overhead, making the problem worse.

Solution:

# Enable TCP MSS clamping to handle MTU issues
sysopt connection tcpmss 1200

# Alternative: Lower the interface MTU
interface outside
 mtu 1300

# For group policy, set explicit MTU
group-policy ANYCONNECT_POLICY attributes
 vpn-tunnel-protocol ssl-client
 address-pools value VPN_POOL
 webvpn
  anyconnect mtu 1200

Case Study 3: Intermittent Tunnel Failures

Problem Description:

IPsec tunnel works perfectly for days, then suddenly fails. Restarting the tunnel fixes it temporarily, but the problem recurs randomly.

Symptoms:

  • Tunnel shows "DOWN" status randomly
  • No clear pattern to failures (not time-based)
  • Manual restart always fixes the issue
  • Both peers show different failure times

Investigation Process:

# Enable detailed IKE debugging
debug crypto ikev2 platform 255
debug crypto ikev2 protocol 255

# Check for certificate expiration
show crypto ca certificates

# Verify time synchronization
show clock detail
show ntp status

# Monitor DPD activity
show crypto ikev2 sa detail | include DPD
Root Cause:

Intermediate certificate in the CA chain had expired. The tunnel worked when cached certificates were used, but failed during periodic certificate validation.

Solution:

# Update the intermediate CA certificate
crypto ca trustpoint INTERMEDIATE-CA
 enrollment url http://ca.company.com/certsrv
 auto-enroll regenerate

# Set up certificate monitoring
crypto ca trustpoint INTERMEDIATE-CA
 revocation-check crl
 crl optional

# Configure certificate renewal notifications
logging enable
logging host 192.168.1.100
logging trap warnings

Troubleshooting Checklist

Systematic Troubleshooting Steps:

Phase 1 Issues:
  • ✓ Verify UDP 500/4500 connectivity
  • ✓ Check pre-shared keys
  • ✓ Validate IKE policies match
  • ✓ Confirm peer addresses
  • ✓ Check certificate validity
  • ✓ Verify time synchronization
Phase 2 Issues:
  • ✓ Verify proxy-ID matching
  • ✓ Check IPsec proposals
  • ✓ Validate interesting traffic ACLs
  • ✓ Confirm routing tables
  • ✓ Check NAT exemption
  • ✓ Verify firewall policies
Previous: Packet Analysis Next: Chapter 8 - Best Practices