These are the current port roles:

I just have put some basic MST configuration and NTP on the switches.

SW3(config)#ntp server 13.13.13.1
SW3(config)#span mode mst
SW3(config)#span mst 0 prio 16384
SW3(config)#span mst 1 prio 16384
SW3(config)#span mst conf
SW3(config-mst)#name TST       
SW3(config-mst)#revision 1

Verify initial reachability between the routers.

R1#ping 13.13.13.3

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 13.13.13.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms

R2#ping 25.25.25.5

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 25.25.25.5, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms

Now let’s shutdown Gi0/21 on SW3 which is leading to SW2 root port.
Debug spanning-tree events will show the sequence of events.

May  7 20:32:18.975: MST[0]: Fa0/21 state change forwarding -> disabled
May  7 20:32:18.975: MST[0]: updt roles, root port Fa0/21 going down
May  7 20:32:18.975: MST[0]: Fa0/23 is now root port
May  7 20:32:18.975: MST[0]: Fa0/21 state change disabled -> blocking
May  7 20:32:18.975: MST[0]: Fa0/23 state change blocking -> forwarding
May  7 20:32:18.979: MST[0]: sending proposal on Fa0/3
May  7 20:32:18.983: MST[0]: sending proposal on Fa0/5

The switchover is immediate as expected. Now let’s try to simulate passive
error by implementing BPDU filter.

SW3(config-if)#span bpdufilter enable
SW3(config-if)#do sh clock
20:36:14.354 UTC Tue May 7 2013

This is from SW2:

May  7 20:36:20.008: MST[0]: updt roles, information on root port Fa0/21 expired
May  7 20:36:20.008: MST[0]: Fa0/23 is now root port
May  7 20:36:20.008: MST[0]: Fa0/21 state change forwarding -> blocking
May  7 20:36:20.008: MST[0]: Fa0/3 state change forwarding -> blocking
May  7 20:36:20.008: MST[0]: Fa0/5 state change forwarding -> blocking
May  7 20:36:20.008: MST[0]: Fa0/23 state change blocking -> forwarding
May  7 20:36:20.008: MST[0]: Fa0/21 is now designated
May  7 20:36:20.012: MST[0]: sending proposal on Fa0/21
May  7 20:36:20.012: MST[0]: sending proposal on Fa0/3
May  7 20:36:20.012: MST[0]: sending proposal on Fa0/5

So it took roughly 6 seconds which was expected. Because MST runs
RSTP the results are exactly the same. The only thing that’s really different
with MST is that all BPDUs are piggybacked in the CIST (instance 0). If you have
VLANs mapped to instance 0 and there is a change then the other ISTs may have
to recalculate as well.

So using MST is no different than using RPVST+ from a convergence standpoint.
In future posts I will look at running a mix of RPVST+ and MST and see how
they interconnect.

This post assumes you already have a good basic understanding of STP. This
is not an introductory post on STP.

This is the topology being used:

SW1 is the root and ports towards the routers have been configured with VLAN 23
and portfast. I will run NTP to have the clocks properly synchronized. Currently
the port roles look like this:

I will configure the routers in 23.23.23.0/24 subnet and do a ping to verify connectivity.

R2#ping 23.23.23.3

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 23.23.23.3, timeout is 2 seconds:
.!!!!
Success rate is 80 percent (4/5), round-trip min/avg/max = 1/3/4 ms

Working fine so far. Now let’s take a look at some different failure scenarios.
We turn on logging to a buffer to not flood the console. We will be looking at
spanning tree events.

SW1(config)#logging con 6
SW1(config)#logging buff 7
SW1(config)#logging buff 32768
SW1(config)#do debug spanning-tree events
Spanning Tree event debugging is on

What happens when the root port is shutdown? In theory when the carrier detects
that the link is down it should look at alternate BPDU and start to take that
port through the different port states. This should take around 30 seconds.

This is output from SW2.

May  7 10:27:03.314: STP: VLAN0023 new root port Fa0/16, cost 38
May  7 10:27:18.321: STP: VLAN0023 Fa0/16 -> learning
May  7 10:27:33.329: STP: VLAN0023 sent Topology Change Notice on Fa0/16
May  7 10:27:33.329: STP: VLAN0023 Fa0/16 -> forwarding

The timing is almost perfect. The port goes through listening and learning
at 15 seconds each before it goes to forwarding almost exactly 30 seconds after
the port was shutdown.

What happens when there is an indirect failure? The switch has to expire the root BPDU
before it believes other BPDUs with worse cost. This should take around 20 seconds. By
default Maxage will be set to 20 seconds.

SW1#sh span | i Age
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
SW2#sh span int f0/13 det | i age
   Timers: message age 1, forward delay 0, hold 0

We will this time simulate a passive error by configuring BPDU filter on SW1 towards
SW2.

SW1(config-if)#span bpdufilter enable   
SW1(config-if)#do sh clock
10:39:05.598 UTC Tue May 7 2013

This has created a bridging loop but in this case we just want to see how long it
takes before the alternate port is coming up as root.

May  7 10:39:24.046: STP: VLAN0023 new root port Fa0/16, cost 38
May  7 10:39:24.046: STP: VLAN0023 Fa0/16 -> listening
May  7 10:39:39.053: STP: VLAN0023 Fa0/16 -> learning
May  7 10:39:54.061: STP: VLAN0023 sent Topology Change Notice on Fa0/16
May  7 10:39:54.061: STP: VLAN0023 Fa0/16 -> forwarding

So it took almost 20 seconds for the BPDU to expire. Then the port goes through
the ordinary state changes. Roughly 48.5 seconds after the filter was applied
the port went into forwarding. For passive failures when running PVST+ the
maximum recovery time should be 50 seconds.

Now let’s look at PVST+ with Uplinkfast configured. The theory is that when a
root port fails the Alternate port should be bypass listening and learning
states and go direct to forwarding. Let’s try this out.

SW2(config)#spanning-tree uplinkfast

May  7 10:46:37.260: STP: VLAN0023 new root port Fa0/16, cost 3038
May  7 10:46:38.249: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/13, changed state to down
May  7 10:46:39.264: %LINK-3-UPDOWN: Interface FastEthernet0/13, changed state to down
May  7 10:46:39.264: STP: VLAN0023 sent Topology Change Notice on Fa0/16

It took only 2 seconds from realizing the port was down to putting the alternate
port into forwarding. For PVST+ this is a great enhancement. What if there is
a passive error?

SW1(config-if)#span bpdufilter enable
SW1(config-if)#do sh clock
10:51:11.870 UTC Tue May 7 2013

May  7 10:51:30.216: STP: VLAN0023 new root port Fa0/16, cost 3038
May  7 10:51:30.216: STP: VLAN0023 sent Topology Change Notice on Fa0/16

There is nothing to be done about the Maxage expiring but the port is
brought up after that. So instead of 50 seconds it takes about 20 seconds.

That’s it for PVST+. Now let’s move on to RPVST+. RPVST+ works by synchronizing
the topology and it has optimizations builtin. If a port fails then it should
converge almost instantly.

May  7 10:56:34.421: RSTP(1): updt roles, root port Fa0/13 going down
May  7 10:56:34.421: RSTP(1): Fa0/16 is now root port
May  7 10:56:34.421: RSTP(1): syncing port Fa0/4
May  7 10:56:34.421: RSTP(1): syncing port Fa0/6
May  7 10:56:34.421: RSTP(1): syncing port Fa0/24
May  7 10:56:34.421: RSTP(23): updt roles, root port Fa0/13 going down
May  7 10:56:34.421: RSTP(23): Fa0/16 is now root port
May  7 10:56:34.438: RSTP(1): transmitting a proposal on Fa0/4
May  7 10:56:34.438: RSTP(1): transmitting a proposal on Fa0/6
May  7 10:56:34.438: RSTP(1): transmitting a proposal on Fa0/24
May  7 10:56:35.419: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/13, changed state to down
May  7 10:56:35.578: RSTP(1): transmitting a proposal on Fa0/4
May  7 10:56:35.578: RSTP(1): transmitting a proposal on Fa0/6
May  7 10:56:35.578: RSTP(1): transmitting a proposal on Fa0/24
May  7 10:56:36.434: %LINK-3-UPDOWN: Interface FastEthernet0/13, changed state to down

It instantly failovers to the Alternate port and then starts synchronizing
the topology by sending out proposals. What if there was a passive failure?
In theory after RPVST+ misses 3 BPDUs it should realize that it needs to
start using the alternate path. Let’s try it out.

SW1(config-if)#span bpdufilter enable
SW1(config-if)#do sh clock
11:01:12.960 UTC Tue May 7 2013

May  7 11:01:16.648: RSTP(1): Fa0/13 rcvd info expired
May  7 11:01:16.648: RSTP(1): updt roles, information on root port Fa0/13 expired
May  7 11:01:16.648: RSTP(1): Fa0/16 is now root port
May  7 11:01:16.648: RSTP(1): Fa0/13 blocked by re-root
May  7 11:01:16.648: RSTP(1): syncing port Fa0/4
May  7 11:01:16.648: RSTP(1): syncing port Fa0/6
May  7 11:01:16.648: RSTP(1): syncing port Fa0/24
May  7 11:01:16.648: RSTP(1): Fa0/13 is now designated
May  7 11:01:16.648: RSTP(23): Fa0/13 rcvd info expired
May  7 11:01:16.648: RSTP(23): updt roles, information on root port Fa0/13 expired
May  7 11:01:16.648: RSTP(23): Fa0/16 is now root port
May  7 11:01:16.648: RSTP(23): Fa0/13 blocked by re-root
May  7 11:01:16.648: RSTP(23): Fa0/13 is now designated

Already around 4 seconds later the topology has converged. It should take
maximum 6 seconds depending on when the last BPDU was received before the
failure.

As you can see it’s very important to detect carrier down. If you do detect it
and are running RPVST+ then convergence is almost immediate. So when designing your
network try to avoid use fiber converts and such that won’t shut down the RJ45 side
if the optical goes down. Designing for convergence is just not about protocols, you
also need to consider the physical infrastructure.

I hope this post has given you a good insight to the convergence of STP.

Who is Ruhann?

Ruhann du Plessis 2x CCIE #24163 (RS, SP) is an experienced engineer
that designs and works with large MPLS VPN networks, intra/inter-AS
routing, large data centers and so on.

The book was written to be used as a kind of quick reference. You
will find both theory but must important config sets that describe
how to configure the different features. Relevant show commands
and how to troubleshoot is also shown which is really good. Also links
to the DOCCD are included so that it becomes easy to find where all
features are located.

The book starts by describing a feature/protocol with some theory and
facts, often in bullet point form. On top of the page there is a
reference to the DOCCD to find the relevant feature. Then the config set
shows how to configure the feature and finally show commands and how
to troubleshoot is shown at the end of the section. There is also a
reference to relevant RFCs describing the features/protocols.

From what I’ve seen this book looks great! The RS book is a great help
in passing the RS lab and now there is an equally good book to help
in passing the SP lab as well.

I really like to use the book as a reference. It’s sometimes easier to
find the information the the handbook than going to the Cisco documentation.
The config sets are even better then what is shown in the Cisco docs.

There is a sample available of the SP handbook here.

To buy it go to Ruhanns site. It’s only 98$.

One important thing to know about NX-OS is that features are selectively enabled.
This means that if you are not running OSPF then there is no need to have that
process running. We can check what features are running.

N7K-1# sh feature | ex not | grep enabled
hsrp_engine           1         enabled 
sshServer             1         enabled 
vtp                   1         enabled

As you can see NX-OS has some nice features like grep which is a nice addition
to regular IOS. There are also additional things that can be done like sort, count
and count unique instances.

N7K-1# sh feature | ex not | grep enabled | count
3

By default Telnet is not enabled which is good. It’s more secure to use SSH.
If we want to add it we can do it with the feature command.

N7K-1(config)# feature telnet
N7K-1# sh feature | grep telnet
telnetServer          1         enabled

In regular IOS we limit the number of VTY sessions with the line vty command.
In NX-OS the session-limit command is used instead.

N7K-1# conf t
Enter configuration commands, one per line.  End with CNTL/Z.
N7K-1(config)# line vty
N7K-1(config-line)# session-limit 5

SSH is enabled by default. A crypto key should already be generated or you can generate
a new one.

N7K-1(config)# ssh key rsa 1024 force
deleting old rsa key.....
generating rsa key(1024 bits).....
.
generated rsa key

With the show users command we can see from which TTYs the users are logged in.

N7K-1# sh users
NAME     LINE         TIME         IDLE          PID COMMENT
admin    pts/0        Apr 30 05:22   .         21294 (10.20.30.200)
admin    pts/1        Apr 30 05:28   .         21845 (10.20.30.200) session=ssh *

When logging in to a NX-OS device the user goes straight to exec mode. There
is no need to enable. There are 4 different types of accounts available in NX-OS.
These are:

• network-admin—Complete read-and-write access to the entire Cisco NX-OS device (only available in the default VDC)
• network-operator—Complete read access to the entire Cisco NX-OS device (only available in the default VDC)
• vdc-admin—Read-and-write access limited to a VDC
• vdc-operator—Read access limited to a VDC

This makes it easy to create users that should have only read access.

N7K-1(config)# username daniel password daniel role network-operator
login: daniel
Password: 
Last login: Mon Apr 29 18:56:23 from 10.20.30.200
Cisco NX-OS Software
N7K-1# conf t
Enter configuration commands, one per line.  End with CNTL/Z.
N7K-1(config)# router ospf 1
% Permission denied for the role
N7K-1(config)#

SNMP should be enabled for network management. SNMP version 2C or 3 can
be enabled.

N7K-1(config)# snmp-server community public ro
N7K-1# show snmp community
Community            Group / Access      context    acl_filter
---------            --------------      -------    ----------
public                network-operator           

For more secure SNMP setup version 3 should be used. SNMPv3 can be setup to use
authentication or authentication and encryption. By default the users we create
will be created as SNMP users also which makes the configuration simple.

N7K-1# show snmp user
______________________________________________________________
                  SNMP USERS 
______________________________________________________________

User                          Auth  Priv(enforce) Groups                        
____                          ____  _____________ ______                        
daniel                        md5   des(no)       network-operator

New users can be created as well.

N7K-1# conf t
Enter configuration commands, one per line.  End with CNTL/Z.
N7K-1(config)# snmp-server user SNMPadmin ?

  WORD   Group name (ignored for notif target user) (Max Size 28)
  auth   Authentication parameters for the user

N7K-1(config)# snmp-server user SNMPadmin auth ?
  md5  Use HMAC MD5 algorithm for authentication
  sha  Use HMAC SHA algorithm for authentication

N7K-1(config)# snmp-server user SNMPadmin auth md5 ?
  WORD  Authentication password for user (Max Size 130)

N7K-1(config)# snmp-server user SNMPadmin auth md5 admin ?

  engineID      EngineID for configuring notif target user (for V3 informs)
  localizedkey  Specifies whether the passwords are in localized key format
  priv          Encryption parameters for the user

N7K-1(config)# snmp-server user SNMPadmin auth md5 admin priv ?
  WORD     Privacy password for user (Max Size 130)
  aes-128  Use 128-bit AES algorithm for privacy

N7K-1(config)# snmp-server user SNMPadmin auth md5 admin priv aes-128 ?
  WORD  Privacy password for user (Max Size 130)

N7K-1(config)# snmp-server user SNMPadmin auth md5 admin priv aes-128 secret ?

  engineID      EngineID for configuring notif target user (for V3 informs)
  localizedkey  Specifies whether the passwords are in localized key format

N7K-1(config)# snmp-server user SNMPadmin auth md5 admin priv aes-128 secret 
user password must be atleast 8 characters
N7K-1(config)# snmp-server user SNMPadmin auth md5 admin priv aes-128 secret1234 
user password must be atleast 8 characters
N7K-1(config)# snmp-server user SNMPadmin auth md5 admin1234 priv aes-128 secret1234

The password must be at least 8 characters. To enforce all SNMPv3 PDUs to be
authenticated and encrypted the following command is used.

N7K-1(config)# snmp-server globalEnforcePriv 
N7K-1(config)#

Now to see that it works.

N7K-1# sh snmp user
______________________________________________________________
                  SNMP USERS [global privacy flag enabled]
______________________________________________________________

User                          Auth  Priv(enforce) Groups                        
____                          ____  _____________ ______                        
daniel                        md5   des(no)       network-operator              

SNMPadmin                     md5   aes-128(no)   network-operator    

And there you have it. A basic look at the management setup of NX-OS. More
posts will follow.

In the future I will try to blog more about datacenter technologies like CSR and Nexus1kv.
If you have something you want covered post in comments and I’ll have a look at it. Thanks
for reading!

Daniel Dib
CCIE #37149

So to start you should have ESXi 5.0. If you have an enterprise version of ESX
that is great but I don’t so I’m using ESXi. I am managing it via the vSphere client.

To install the CSR you can go to Cisco CSR config guide or read the guide by Brian Dennis at INE

I have installed Ubuntu desktop 12.10 64-bit version. You will need some tools to
have a good setup. I recommend you install the following:

Dynamips
Dynagen
XRDP
Wireshark
Gnome-fallback
Vmware tools
Screen

sudo apt-get install dynamips
sudo apt-get install dynagen
sudo apt-get install xrdp
sudo apt-get install wireshark
sudo apt-get install open-vm-tools
sudo apt-get install screen
sudo apt-get install gnome-session-fallback
cd ~
touch .xsession
echo gnome-session --session=gnome-fallback > .xsession

You can then use RDP to connect to the Ubuntu machine. If you don’t need the graphics you
can use use SSH as usual.

I will put together a topology that looks like this:

As you can see I will be using 3 VLANs. One VLAN is for managing the devices.
I can login to the CSRs and the Dynamips routers from this network. The CSRs
uses GigabitEthernet0 by default as a management interface that are placed
in the VRF Mgmt-intf.

You can use a dedicated vSwitch or create the VLANs on the standard vSwitch. I have
just created VLANs in the regular vSwitch. You configure this under Inventory -> Configuration
-> Networking -> Add networking

After clicking “Add Networking” choose connection type “Virtual machine”

Choose an existing vSwitch or create a new one if you wish.

Then choose the name for your network and assign a VLAN ID to it. You can use the same
numbers I did or choose something else.

Finish the guide and the new network will be present. We want to edit a setting
for the networks that will connect to Dynamips. We want to set the port group to
promiscous mode so that CDP frames and other traffic not destined to the VM can
arrive to the VMs. This will create some overhead but shouldn’t be an issue in
a lab network. Click “Properties…” for the vSwitch.

Select the network and choose “Edit…” then under the Security tab set
“Promiscous Mode:” to Accept.

After creating all the networks they need to be assigned to the virtual machines.
For the CSRs the GigabitEthernet0 will be assigned to the MGMT network and Gi1 to
CSR to Dynamips 1 and Gi2 to CSR to Dynamips2.

Right click the VM and choose “Edit Settings…”. The NICs should be assigned like this:

Do the same also for the Dynamips VM. In theory there should now be connectivity.
We will use a topology that looks like this:

We need to create a .net file that can be used to create this topology.
5 routers will be running in Dynamips so 1 or 2 hypervisors should be enough.
As usual you need to find suitable Idle-PC value for your topology. My .net
looks like this.

autostart = False
[127.0.0.1:7200]
	workingdir = /home/daniel/dynamips/working/CSR
	udp = 10000
	[[7200]]
        	image = /home/daniel/IOS/c7200-adventerprisek9-mz.150-1.M1.bin-unpacked
        	ram = 256
        	idlepc = 0x628cc49c
        	ghostios = True
	[[ROUTER R1]]
        	model = 7200
        	console = 20061
        	f1/0 = R2 f1/0
		f1/1 = R3 f1/0
		f2/0 = nio_gen_eth:eth1
	[[ROUTER R2]]
        	model = 7200
        	console = 2002
        	f1/0 = R1 f1/0
		f1/1 = R4 f1/0
	[[ROUTER R3]]
        	model = 7200
        	console = 2003
        	f1/0 = R1 f1/1
		f1/1 = R4 f1/0
	[[ROUTER R4]]
        	model = 7200
        	console = 2004
        	f1/0 = R2 f1/1
		f1/1 = R3 f1/1
	[[ROUTER R5]]
        	model = 7200
        	console = 2005
        	f1/0 = nio_gen_eth:eth2
	

The only thing special here is that R1 and R5 are connecting to the outside
world. By using the generic NIO descriptor we are connecting to the Ethernet
interfaces leading to the VM networks.

It’s time to start the Dynamips process. I will use screen because I want to
keep the process running even if I disconnect my session.

daniel@Dynamips:~/.gns3$ sudo screen -mS dynamips dynamips -H 7200 &
daniel@Dynamips:~/.gns3$ dynagen CSR3.net

I have started all devices so I should be able to reach them and configure them now.

I will configure routers R1-R4 to run OSPF. R4 will announce its loopback 4.4.4.4
and this should be reachable from R5 on the other side of the network.
R1 will run BGP to both CSR1 and 2. This is the configuration applied to R1.

interface FastEthernet1/0
 ip address 12.12.12.1 255.255.255.0
 ip ospf 1 area 0
 duplex auto
 speed auto
 !
!
interface FastEthernet1/1
 ip address 13.13.13.1 255.255.255.0
 ip ospf 1 area 0
 duplex auto
 speed auto
 !
!
interface FastEthernet2/0
 ip address 10.10.10.1 255.255.255.0
 duplex auto
 speed auto
 !
!
interface FastEthernet2/1
 no ip address
 shutdown
 duplex auto
 speed auto
 !
!
router ospf 1
 router-id 1.1.1.1
 log-adjacency-changes
 redistribute bgp 1 subnets
!
router bgp 1
 no synchronization
 bgp router-id 1.1.1.1
 bgp log-neighbor-changes
 redistribute ospf 1
 neighbor 10.10.10.11 remote-as 100
 neighbor 10.10.10.12 remote-as 100
 no auto-summary

Configuration for CSRs is very simple.

interface GigabitEthernet1
 ip address 10.10.10.11 255.255.255.0
 negotiation auto
!
interface GigabitEthernet2
 ip address 20.20.20.1 255.255.255.0
 ip ospf 1 area 0
 negotiation auto
!
interface GigabitEthernet0
 vrf forwarding Mgmt-intf
 ip address dhcp
 negotiation auto
!
router ospf 1
 redistribute bgp 100 subnets
!
router bgp 100
 bgp log-neighbor-changes
 redistribute ospf 1
 neighbor 10.10.10.1 remote-as 1

CSR2 only has different addressing. Now do we see any routes?

CSR1#sh bgp ipv4 uni
BGP table version is 7, local router ID is 20.20.20.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
 *>  4.4.4.4/32       10.10.10.1               3             0 1 ?
 *>  12.12.12.0/24    10.10.10.1               0             0 1 ?
 *>  13.13.13.0/24    10.10.10.1               0             0 1 ?
 *>  20.20.20.0/24    0.0.0.0                  0         32768 ?
 *>  24.24.24.0/24    10.10.10.1               2             0 1 ?
 *>  34.34.34.0/24    10.10.10.1               2             0 1 ?

Looks good. Now let’s just verify that R5 sees them as well.

R5#sh ip route ospf | be Gate
Gateway of last resort is not set

      4.0.0.0/32 is subnetted, 1 subnets
O E2     4.4.4.4 [110/1] via 20.20.20.2, 00:03:33, FastEthernet1/0
                 [110/1] via 20.20.20.1, 00:05:18, FastEthernet1/0
      12.0.0.0/24 is subnetted, 1 subnets
O E2     12.12.12.0 [110/1] via 20.20.20.2, 00:03:33, FastEthernet1/0
                    [110/1] via 20.20.20.1, 00:05:18, FastEthernet1/0
      13.0.0.0/24 is subnetted, 1 subnets
O E2     13.13.13.0 [110/1] via 20.20.20.2, 00:03:33, FastEthernet1/0
                    [110/1] via 20.20.20.1, 00:05:18, FastEthernet1/0
      24.0.0.0/24 is subnetted, 1 subnets
O E2     24.24.24.0 [110/1] via 20.20.20.2, 00:03:33, FastEthernet1/0
                    [110/1] via 20.20.20.1, 00:05:18, FastEthernet1/0
      34.0.0.0/24 is subnetted, 1 subnets
O E2     34.34.34.0 [110/1] via 20.20.20.2, 00:03:33, FastEthernet1/0
                    [110/1] via 20.20.20.1, 00:05:18, FastEthernet1/0

ECMP is implemented since the cost is the same to ASBRs.
Final test is to ping 4.4.4.4.

R5#ping 4.4.4.4

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 4.4.4.4, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 16/20/24 ms

And it works! Pretty cool stuff. So now we have a lab with both IOS and IOS-XE.
If we had real switches we could throw this into the topology as well.
If you have enterprise ESX you could even practice Nexus with N1kv image.
So you see that we can build some great topologies here.

This does take some computing power, mostly RAM. If I check ESXi I see that
the Ubuntu box is using around 2GHz CPU and about 2GB RAM. The CSRs are using
each 750MHz CPU and 3GB RAM. So in total you are looking at around 4GHz CPU
and 8GB RAM. It is doable on a well equipped laptop.

As most of my readers know by now I like to help people get started with their
careers and help them along with their studies. I’ve been quite active on the
Cisco Learning Network lately and also I have now started to write technical
articles to prepare students for the CCNA. These articles will be published
by Intense School which is a training company.

Most of you here might already be past CCNA level but I’ll link to my articles
anyway in case you want to read it or if you have friends studying for the CCNA.

The first one is about IP access-lists and you can find it at Intense school.

It is already well known that Cisco uses something called IOS on Unix (IOU)
in the CCIE lab. It is a virtualized IOS running on Solaris and can run
both routing and switching. In the future I think the entire CCIE lab will
be virtualized.

Cisco has also lately been releasing IOS-XE virtualized which is called CSR.
It is the Cisco Cloud Services Router (CSR). You can now get this in a VM
and it’s called CSR1000v. This is great that Cisco is moving in this direction.

Now, not everyone may know that internally Cisco has been running virtualized
XR for a couple of years and I’ve seen it referred to as XR4U. I’m not sure if
that is the official name but now the rumour is that this XR4U might be released
to the public in a VM of some sorts. This would be very big news if true as people
have difficulties finding rack time on XR devices and would certainly be a major
deal for anyone wanting to go for their CCIE SP. I would expect this VM to have
some kind of limitations just like the CSR1000v.

If you read this PDF it is mostly talk about SDN
which may be interesting in itself but the real interesting thing to me is on page 30 and 31
where the Cisco VIRL is introduced. Seems like there should be VMs available for both
IOS XE, IOS XR, IOS and NXOS.

NXOS is available today through the Nexus1000v.

I’m trying to get some more information from Cisco. If this is true it is major news
and could help anyone wanting to learn Cisco do this in a much easier way without
using tools like GNS3 and Packet Tracer etc.

First of all, what is the point of having a forwarding address? Look at the topology
below.

R3 is the only one running BGP to R4. If the FA is not set then there will be an
extra hop compared to R2 sending the traffic directly to R4.

R1#sh ip route 10.10.4.0
Routing entry for 10.10.4.0/24
  Known via "ospf 1", distance 110, metric 1
  Tag 4, type extern 2, forward metric 20
  Last update from 10.10.12.2 on FastEthernet0/0, 00:00:23 ago
  Routing Descriptor Blocks:
  * 10.10.12.2, from 10.10.23.3, 00:00:23 ago, via FastEthernet0/0
      Route metric is 1, traffic share count is 1
      Route tag 4

R1#sh ip ospf data ex 10.10.4.0

            OSPF Router with ID (10.10.12.1) (Process ID 1)

                Type-5 AS External Link States

  Routing Bit Set on this LSA
  LS age: 35
  Options: (No TOS-capability, DC)
  LS Type: AS External Link
  Link State ID: 10.10.4.0 (External Network Number )
  Advertising Router: 10.10.23.3
  LS Seq Number: 80000001
  Checksum: 0xEB7D
  Length: 36
  Network Mask: /24
        Metric Type: 2 (Larger than any link state path)
        TOS: 0 
        Metric: 1 
        Forward Address: 0.0.0.0
        External Route Tag: 4

R1#traceroute 10.10.4.4 num

Type escape sequence to abort.
Tracing the route to 10.10.4.4

  1 10.10.12.2 44 msec 44 msec 32 msec
  2 10.10.23.3 60 msec 36 msec 40 msec
  3 10.10.234.4 84 msec *  76 msec

Because the forwarding address is set to 0 the traffic must flow through the
ASBR originating the LSA.

Which conditions must be met to set the FA?

The interface on the ASBR must have OSPF enabled. It must not be passive and it
must be broadcast. Let’s enable this on R3.

R3#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R3(config)#int f0/1
R3(config-if)#ip ospf 1 area 0

Now check the external LSA on R1 and a traceroute.

R1#sh ip ospf data ex 10.10.4.0

            OSPF Router with ID (10.10.12.1) (Process ID 1)

                Type-5 AS External Link States

  Routing Bit Set on this LSA
  LS age: 243
  Options: (No TOS-capability, DC)
  LS Type: AS External Link
  Link State ID: 10.10.4.0 (External Network Number )
  Advertising Router: 10.10.23.3
  LS Seq Number: 80000002
  Checksum: 0xF66E
  Length: 36
  Network Mask: /24
        Metric Type: 2 (Larger than any link state path)
        TOS: 0 
        Metric: 1 
        Forward Address: 10.10.234.4
        External Route Tag: 4

R1#traceroute 10.10.4.4 num

Type escape sequence to abort.
Tracing the route to 10.10.4.4

  1 10.10.12.2 48 msec 32 msec 64 msec
  2 10.10.234.4 96 msec *  88 msec

The traffic is now flowing directly via R2. The key point here is that in broadcast
networks all routers can communicate with each other (full mesh). We can see this by
looking at the type2 LSA.

R1#sh ip ospf data net 10.10.234.3

            OSPF Router with ID (10.10.12.1) (Process ID 1)

                Net Link States (Area 0)

  Routing Bit Set on this LSA
  LS age: 179
  Options: (No TOS-capability, DC)
  LS Type: Network Links
  Link State ID: 10.10.234.3 (address of Designated Router)
  Advertising Router: 10.10.23.3
  LS Seq Number: 80000001
  Checksum: 0x3485
  Length: 32
  Network Mask: /24
        Attached Router: 10.10.23.3
        Attached Router: 10.10.12.2

Why isn’t a point to point network valid? Well, the name pretty much says it all.
With point-to-point there can only be two routers connected so there is no use
in setting the FA because the traffic must flow through the router originating
the LSA.

If we look at the router LSA from R2 when we have broadcast network type it looks
like this:

R1#sh ip ospf data router 10.10.12.2

            OSPF Router with ID (10.10.12.1) (Process ID 1)

                Router Link States (Area 0)

  LS age: 7
  Options: (No TOS-capability, DC)
  LS Type: Router Links
  Link State ID: 10.10.12.2
  Advertising Router: 10.10.12.2
  LS Seq Number: 8000000A
  Checksum: 0x977B
  Length: 60
  Number of Links: 3

    Link connected to: a Transit Network
     (Link ID) Designated Router address: 10.10.234.3
     (Link Data) Router Interface address: 10.10.234.2
      Number of TOS metrics: 0
       TOS 0 Metrics: 1

    Link connected to: a Transit Network
     (Link ID) Designated Router address: 10.10.23.2
     (Link Data) Router Interface address: 10.10.23.2
      Number of TOS metrics: 0
       TOS 0 Metrics: 10

    Link connected to: a Transit Network
     (Link ID) Designated Router address: 10.10.12.1
     (Link Data) Router Interface address: 10.10.12.2
      Number of TOS metrics: 0
       TOS 0 Metrics: 10

You can see that the 10.10.234.0 is a transit network and then the type 2 LSA shows
which routers are connected and the network mask. Now if we change to point to point.

R1#sh ip ospf data router 10.10.12.2

            OSPF Router with ID (10.10.12.1) (Process ID 1)

                Router Link States (Area 0)

  LS age: 59
  Options: (No TOS-capability, DC)
  LS Type: Router Links
  Link State ID: 10.10.12.2
  Advertising Router: 10.10.12.2
  LS Seq Number: 8000000B
  Checksum: 0xF2E3
  Length: 72
  Number of Links: 4

    Link connected to: another Router (point-to-point)
     (Link ID) Neighboring Router ID: 10.10.23.3
     (Link Data) Router Interface address: 10.10.234.2
      Number of TOS metrics: 0
       TOS 0 Metrics: 1

    Link connected to: a Stub Network
     (Link ID) Network/subnet number: 10.10.234.0
     (Link Data) Network Mask: 255.255.255.0
      Number of TOS metrics: 0
       TOS 0 Metrics: 1

    Link connected to: a Transit Network
     (Link ID) Designated Router address: 10.10.23.2
     (Link Data) Router Interface address: 10.10.23.2
      Number of TOS metrics: 0
       TOS 0 Metrics: 10

    Link connected to: a Transit Network
     (Link ID) Designated Router address: 10.10.12.1
     (Link Data) Router Interface address: 10.10.12.2
      Number of TOS metrics: 0
       TOS 0 Metrics: 10

The 10.10.234.0 network is now a stub network which means it can’t be used for transit.
Usually there should only be two routers connected here, we shouldn’t use P2P network
type if there is an Ethernet segment with multiple routers.

So finally why is P2MP not valid? Because P2MP is used in NBMA networks. These networks
are usually partially meshed and from the perspective of OSPF it is a collection of
point to point links. This is how the LSA looks.

R1#sh ip ospf data router 10.10.12.2

            OSPF Router with ID (10.10.12.1) (Process ID 1)

                Router Link States (Area 0)

  LS age: 8
  Options: (No TOS-capability, DC)
  LS Type: Router Links
  Link State ID: 10.10.12.2
  Advertising Router: 10.10.12.2
  LS Seq Number: 8000000D
  Checksum: 0xFCD6
  Length: 72
  Number of Links: 4

    Link connected to: another Router (point-to-point)
     (Link ID) Neighboring Router ID: 10.10.23.3
     (Link Data) Router Interface address: 10.10.234.2
      Number of TOS metrics: 0
       TOS 0 Metrics: 1

    Link connected to: a Stub Network
     (Link ID) Network/subnet number: 10.10.234.2
     (Link Data) Network Mask: 255.255.255.255
      Number of TOS metrics: 0
       TOS 0 Metrics: 0

    Link connected to: a Transit Network
     (Link ID) Designated Router address: 10.10.23.2
     (Link Data) Router Interface address: 10.10.23.2
      Number of TOS metrics: 0
       TOS 0 Metrics: 10

    Link connected to: a Transit Network
     (Link ID) Designated Router address: 10.10.12.1
     (Link Data) Router Interface address: 10.10.12.2
      Number of TOS metrics: 0
       TOS 0 Metrics: 10

It looks very similar to P2P with the difference that the stub network has a mask
of /32. This is useful in partial mesh where spokes need to reach each other via
the hub and don’t have a DLCI between them.

So it only makes sense to use FA in broadcast networks because that is the only
place where routers are guaranteed to be able to communicate to each other because
it is by nature fully meshed.

Here are some interesting sources for operation of v6.

draft-matthews-v6ops-design-guidelines-01

This document discusses if IPv4 and IPv6 traffic
should be mixed on the same interface or should different interfaces be used? Should
link local or global addressing be used for routing? Should v6 routes be transferred
over v4 in BGP sessions?

draft-ietf-v6ops-enterprise-incremental-ipv6-01

This document is for deploying v6 in an enterprise network. Things like security policy,
addressing plan and IPv6 myths are brought up.

draft-ietf-opsec-lla-only-01

This document is purely about the advantages and disadvantages of only running link local
addresses.

Also, don’t miss out on information that is freely available at Cisco Live. Here are
some interesting sessions on IPv6 from Melbourne.

BRKRST-2301 – Enterprise IPv6 Deployment (2013 Melbourne)
BRKRST-1069 – Understanding IPv6 (2013 Melbourne)
ITMGEN-1313 – Preparing for IPv6 in the Enterprise (2013 Melbourne)
BRKRST-2311 – IPv6 Planning, Deployment and Troubleshooting (2013 Melbourne)
BRKSEC-2003 – IPv6 Security Threats and Mitigations (2013 Melbourne)
COCRST-2464 – Inside Cisco IT: Making The Leap To IPv6 (2013 Melbourne)

As you can see. IPv6 is a pretty big deal these days at Cisco Live. Then you also have
books, configuration guides etc but this should give you a good start to see what challenges
and considerations you should have when deploying IPv6.