29 November 2010

NAT: Common Issues

Some of the most common issues when implementing NAT in the field are described next. Most of these are common configuration mistakes, but some of them are due to a lack of understanding of how NAT is supposed to work.

The symptoms of most of these problems are similar, meaning that the devices, whose IP addresses are NATed, are unable to access the public network as they are supposed to. Most of these issues can be diagnosed by understanding what goal NAT is being implemented to achieve and then looking at the configuration to see if one of the following mistakes is the source of the problem:
  • Access lists blocking NATed or non-NATed traffic- It is important to keep in mind NAT's operation in relation to the access control lists. Opening an access list for non-NATed traffic, when in reality the traffic hitting it is NATed traffic, causes traffic to be dropped. For instance, ACLs on the interface that has ip nat inside configured on it should use the non-NATed addressing for traffic entering it from the private netwok, and ACLs on the interface that has ip nat outside configured on it should use NATed addressing for traffic entering it from the public side. 
  • NAT access list is missing networks to be NATed- The access list that is used to define interesting traffic for the NAT operation must define all the network ranges that need to be NATed. Missing an address or a number of addresses from this access list keeps NAT from occurring for the traffic coming from these addresses. 
  • Missing the overload keyword in the NAT statement- If only a limited number of globally routable addresses are available to do NAT, Port Address Translation (PAT) or Overload NAT must be used to allow all the hosts on the RFC 1918 addressed network to reach the public network. To set up PAT, the keyword overload must be used at the end of the NAT configuration command. Omitting this command causes PAT not to be turned on, resulting in only a limited number of hosts being able to access the public network or the Internet. Here is an example of using the overload keyword: ip nat inside source access list 1 interface Ethernet0/0 overload 
  • Asymmetric routing causes NAT to fail- NAT occurs when a packet enters an interface configured with the ip nat inside command and leaves via an interface configured with the ip nat outside command. On a router with multiple interfaces, you must make sure that all interfaces on which traffic, that needs to be NATed, enters the router, and all interfaces from which this traffic exits, are configured with the ip nat inside and ip nat outside commands, respectively. Failure to do so causes NAT not to occur for traffic passing through the interface that does not have the appropriate NAT command configured on it. 
  • Overlapping addresses in the NAT pool and static NAT entries- It is important to make sure that the IP addresses in the NAT pool are not also being used for a static NAT translation. This can cause NAT to fail intermittently. 
  • Using extended access lists to define interesting traffic for NAT- By default, the translations that NAT creates contain only entries for the inside global and inside local address unless PAT is being used. This is done because this is all the information that is needed for simple NAT of inside local addresses to inside global addresses and to de-NAT the return traffic. However, this remains true even when extended access lists are used to define the interesting traffic.

    NAT: Show Commands

    NAT show Commands

    The command most often used to troubleshoot NAT is show ip nat translations
    Sample Output for the show ip nat translations Command
    Router#show ip nat translations
    Pro   Inside global         Inside local         Outside local     Outside global

    Another useful command is show ip nat translations verbose 
    Sample Output for the show ip nat translations verbose Command
    Router#show ip nat translations verbose
    Pro Inside global           Inside local        Outside local    Outside global
                  create 00:00:02,       use 00:00:00,  flags:  extended
                  create 00:01:13,       use 00:00:50,  flags:  extended

    Here is an explanation of the additional fields in this command:
    • create- How long ago the entry was created (in hours:minutes:seconds).
    • use- How long ago the entry was last used (in hours:minutes:seconds).
    • flags- Indicates the type of translation. Possible flags are
      extended- Extended translation
      static- Static translation
      destination- Rotary translation
      outside- Outside translation
      timing out- Translation is no longer used due to a TCP FIN or RST

    In general, the following is the terminology used in the show commands to display various addresses:
    • Inside local address- The IP address that is assigned to a host on the inside network. The address is probably not a legitimate IP address assigned by the Network Information Center (NIC) or service provider. This is typically an RFC 1918 address.
    • Inside global address- A legitimate IP address (assigned by the NIC or service provider) that represents (after translation) one or more inside local IP addresses to the outside world.
    • Outside local address- The IP address of an outside host as it appears to the inside network. Not necessarily a legitimate address, it can be allocated from address space routable on the inside.
    • Outside global address- The IP address assigned to a host on the outside network by the host's owner. The address was allocated from a globally routable address or network space.
    show ip nat statistics is another useful command that gives the administrator information about NAT statistics. 
    Sample Output for the show ip nat statistics Command
    Router#show ip nat statistics
    Total translations: 2 (0 static, 2 dynamic; 0 extended)
    Outside interfaces: Serial0
    Inside interfaces: Ethernet1
    Hits: 135  Misses: 5
    Expired translations: 2
    Dynamic mappings:
    -- Inside Source
    access-list 1 pool net-208 refcount 2
     pool net-208: netmask
            start end
            type generic, total addresses 14, allocated 2 (14%), misses 0

    To summarize, the two relevant NAT show commands are
    • show ip nat translations or show ip nat translations verbose
    • show ip nat statistics

      NAT: Debugs

      NAT Debugs

      The most useful command that can be used to troubleshoot NAT is
      debug ip packet [access-list | detail]
      This debug command essentially shows all the packets that hit the access list for which this command is used. With a properly configured access list, it is often very easy to figure out what is happening to packets as they get routed through the router and where they are headed.

      Another useful NAT debugging command is
      debug ip nat
      This command shows the translations being created on the router as well as packets that are being translated using the NAT translations.

      So the two most important NAT debugs are

      debug ip packet [access-list | detail]
      debug ip nat [access-list | detail]



      interface Ethernet0
      ip address
      ip nat inside

      interface Ethernet1
      ip address
      ip nat outside

      ip nat pool letmeout pre 24
      ip nat inside source list 7 pool letmeout

      ip route
      access-list 7 permit

      Debug Commands

      # debug ip packet
      # debug ip nat

      Debug Outputs

      The output shown below is from turning on the debug commands listed earlier.
      IP: NAT enab = 1 trans = 0 flags = 80

      The NAT debug output below shows a translation for to
      taking place for the traffic headed for the outside local/global address The traffic as can be seen from the IP debugs is ICMPs.
      NAT: s=>, d= [30]
      IP: s= (Ethernet0), d= (Ethernet1), g=, len 100, 
        forward ICMP type=8, code=0

      IOS NAT: Order of Operations

      Understand NAT Order of Operations in IOS
      Inside to Outside

      1. If IPsec, check input access list
      2. Decryption for CET (Cisco Encryption Technology) or IPsec
      3. Check input access list
      4. Check input rate limits
      5. Input accounting
      6. Inspect
      7. Policy routing
      8. Routing
      9. Redirect to web cache
      10. NAT inside to outside (local-to-global translation)
      11. Crypto (check map and mark for encryption)
      12. Check output access list
      13. Inspect
      14. TCP intercept
      15. Encryption
      Outside to Inside

      1. If IPsec, check input access list
      2. Decryption for CET or IPsec
      3. Check input access list
      4. Check input rate limits
      5. Input accounting
      6. Inspect
      7. NAT outside to inside (global-to-local translation)
      8. Policy routing
      9. Routing
      10. Redirect to web cache
      11. Crypto (check map and mark for encryption)
      12. Check output access list
      13. Inspect
      14. TCP intercept
      15. Encryption

      Notice the reverse of routing and natting depending on the flow of the traffic

      PIM and IGMP

      Multicast data, say for audio or video streams, has unicast source IP's and destination IP's in the range The data payload is currently always UDP. Local multicast routing decisions are based on IGMP (protocol 2) and PIM (protocol 103).

      Clients send IGMP join messages to an upstream device with TTL 1, which processes them, adds the multicast destination IP to its list of subscribed groups, and emits in turn a new IGMP message toward it's own upstream device. On a LAN, ethernet packets will use special multicast destination addresses which repeat a chunk of the multicast group IP.

      Within an autonomous system, PIM (protocol independent routing, sparse mode) has a centralized RP (rendezvous point) which keeps track of where multicast senders are for joined group clients. Between autonomous systems, multicast routes are advertised using some other protocol, typically extensions to BGP. Eventually the chain of join messages reaches the multicast source, which starts sending traffic back down the chain toward the client.

      IGMP messages stop as soon as you run into a PIM enabled router. From there the PIM router sends join messages towards the RP or source and a distribution tree gets built. IGMP only goes beyond one hop when IGMP forwarding/proxying is enabled (which is described later). Some consumer level Linksys routers also employ an IGMP proxy to allow multicast traffic to penetrate within a local NAT, when configured to do so.

      Each intermediate router, firewall, or other device has to keep track of which interfaces have multicast clients on them and send subscribed group traffic to only those interfaces. When a client is done, it sends an IGMP leave message upstream, and the traffic stops. If a client simply vanishes, the upstream device is periodically doing IGMP queries to see what groups are active on a particular interface/LAN. Each active client has to respond with an IGMP group membership report packet. Any groups which don't get responses get pruned by the querying device, and the traffic eventually stops.

      The RFC's for multicast and the group assignments by IANA are a bit of a mess; they have mutated a lot over the years, and continue to do so. For example, so far IGMP version 3 exists only in draft form; it hasn't been converted to standards track yet, much less ratified by the IETF. Multicast address assignments are also a bit of a mess. Some particularly important group ranges are:

      • is globally routable and directly assigned by IANA. 
      • is reserved for local LAN stuff. 
      • is the "GLOP" range, where autonomous systems with 16- bit AS's can emit 233.X.Y.0/24 and be globally accessible (clients send joins toward the AS, inside the AS the actual sources register with their PIM RP). Most remote sources you run across will be in this range, because all other globally routable multicast groups have to go to the formality of a specific assignment by IANA. Each 16-bit AS gets its own /24.
      • is for administratively scoped multicast within an autonomous system (routing domain). Most local sources you run across will be in this range, e.g. our DATN channels.

      30 July 2010

      ASA Outside to Inside NAT

      By default, nat command exempts traffic from inside to outside (high security to equal or low security). If you want traffic from outside to inside to bypass NAT, then add an additional nat command and enter outside to identify the NAT instance as outside NAT. You might want to use outside NAT exemption if you configure dynamic NAT for the outside interface and want to exempt other traffic.

      For example, in order to exempt an inside network when accessing any destination address, enter this command:
      hostname(config)#access-list EXEMPT permit ip any 
      hostname(config)# nat (inside) 0 access-list
      In order to use dynamic outside NAT for a DMZ network, and exempt another DMZ network, enter this command:
      hostname(config)#nat (dmz) 1
        outside dns 
      hostname(config)#global (inside) 1
      hostname(config)#access-list EXEMPT permit ip any 
      hostname(config)#nat (dmz) 0 access-list
      In order to exempt an inside address when accessing two different destination addresses, enter this commands:
      hostname(config)#access-list NET1 permit ip
      hostname(config)#access-list NET1 permit ip 
      hostname(config)#nat (inside) 0 access-list NET1