NAT translation – What happened to my traceroute?

This post will describe something interesting that happens to traceroute
when you are doing NAT. The inspiration for this post came from a
thread at IEOC started by Nick O’neill. He was doing ip nat outside translation but I
have found that the same behavior is true also for inside translations.

We will use this topology:

R1 has a loopback 1.1.1.1/32 on the inside. On the outside this will be known as
100.100.100.100. You can see the networks connecting the routers are 12.12.12.0/24,
23.23.23.0/24 and 34.34.34.0/24. R3 will be the router doing NAT and this is the
configuration of it:

interface FastEthernet0/0
 ip address 23.23.23.3 255.255.255.0
 ip nat inside
!
interface FastEthernet0/1
 ip address 34.34.34.3 255.255.255.0
 ip nat outside
!
ip route 1.1.1.1 255.255.255.255 23.23.23.2
!
ip nat inside source static 1.1.1.1 100.100.100.100

This is not a post describing the basics but make sure that you have routing setup.
I am using static routing here. When doing static NAT like this there is bidirectional
communication setup. Remember that when the outside to inside translation is done NAT
is performed before the routing lookup so we must have a route to 1.1.1.1 on R3.

The command above translates the inside local address to an inside global address.
R4 will traceroute to 100.100.100.100 and we will see what happens.

R4#traceroute 100.100.100.100 num prob 1

Type escape sequence to abort.
Tracing the route to 100.100.100.100

  1 34.34.34.3 28 msec
  2 100.100.100.100 124 msec
  3 100.100.100.100 84 msec

Interesting. Hops 2 and 3 are the same and they have the inside global address as source.
What happened here? The first hop is correct. The ICMP TTL exceeded should come back from
the outgoing interface used to reach 34.34.34.4 which was the source of the UDP traceroute
packet sent. Hop 2 should have been 23.23.23.2.

If we compare it to MPLS this almost looks like some core hiding feature as it does not
reveal the internal addressing of our network. If we look at debug on R3 we will see
what is happening.

R3# debug ip nat det
IP NAT detailed debugging is on

NAT: i: icmp (23.23.23.2, 33435) -> (34.34.34.4, 49164) [1]  
NAT: s=23.23.23.2->100.100.100.100, d=34.34.34.4 [1]
NAT: i: icmp (12.12.12.1, 33436) -> (34.34.34.4, 49165) [2]  
NAT: s=12.12.12.1->100.100.100.100, d=34.34.34.4 [2]

We clearly see that the inside addresses are being translated to 100.100.100.100
even though there is no matching NAT statement. The number 33435 is the source
port number and 49164 is the destination port number.

This is the IOS version used for this lab:

R3#sh ver | i IOS
Cisco IOS Software, 3700 Software (C3725-ADVENTERPRISEK9-M), Version 12.4(15)T10, RELEASE SOFTWARE (fc3)

Now if we try it on mainline IOS instead…

R3#sh ver | i IOS
Cisco IOS Software, 3700 Software (C3725-ADVIPSERVICESK9-M), Version 12.4(25d), RELEASE SOFTWARE (fc1)

R4#trace 100.100.100.100 num prob 1

Type escape sequence to abort.
Tracing the route to 100.100.100.100

  1 34.34.34.3 68 msec
  2 23.23.23.2 164 msec
  3 100.100.100.100 196 msec

As you can see with the mainline IOS we can see the real addresses as they are not
being translated.

Knowing me you know that I like to dig deep to find out what is going on. I reached
out to a contact at Cisco to see if he could find any reference to what is happening
here. It turns out that this is related to bug CSCsu37097. Basically the customer
didn’t want the internal addressing revealed and so this was incorporated into IOS.
The symptom was described as: “Symptom:
Traceroute/ICMP unreachable doesn’t translate properly and can cause security problem.”

The following quote is also available from Cisco: “In Cisco IOS Release 15.1(3)T and
later releases, when you configure the traceroute command, NAT returns the same
inside global IP address for all inside local IP addresses.” The source is this
document.

So basically this was implemented as a security feature for NAT. I hope you
learned something interesting.