Daniels quest for CCIE

Did a final mock before lab and barely failed. I got 71/100. To my defence I did loose at least 3 points due to a bug. I had BGP configured correctly but session would not come up, BB was expecting other AS then the one I was coming from but SG had the same solution.

The TS section had some very stupid tasks. I hope I don’t have to see anything like that on the lab.

The config section went pretty well but lost some few points due to bug and a few points from reading the tasks wrong. I need to cut down on those definately. Everything has to be taken literally. If it says that put interface x in OSPF with /64 mask for IPv6. Then you need to see it as /64 so if it is a loopback you need to announce it as point-to-point.

Good news is I pretty much nailed IGP and redistribution went according to plans. Full reachability! I was also able to finish all tasks on time so I’m at least on the right path.

I’ll post more next week.

Continuing to check things off from the blueprint. Did some ZBFW labbing today. Here are some important stuff to be aware of.

ZBFW is basically a wrapper for CBAC. We create policys between zones and assign interfaces to zones instead of applying CBAC rules to interfaces.

By default all traffic to the self zone will be allowed (router from and to router itself). If we apply policys to self zone then everything is dropped except for the traffic that is explicitly permitted. We need to be aware of this to not mess with the routing if we get such a task at the lab.

The self zone can only inspect TCP, UDP and ICMP but not protocols like telnet and SSH. To work around this we can do a class-map matching an ACL AND the protocol TCP if we are matching telnet traffic.

It’s not very intuitive to see which traffic is dropped. We can turn on logging with ip inspect log drop-pkt. This helps a lot to see which traffic is being dropped.

ZBFW is massive in configuration, you will be typing a lot. It is easy to get confused and mix things. Name things intuitively, name class-maps CM_INSIDE_PROTOCOLS, name policy-maps PM_INSIDE_TO_OUTSIDE or names similar to that. If you don’t you will easily get lost after a while due to the massive config.

Packet counters for ZBFW can’t be trusted, this seems to be due to a bug. Verify by pinging or telneting to create traffic.

Use Notepad when creating the config, it is faster and less prone to errors.

All traffic flows are unidirectional so we need to create zone pairs for both directions depending if we want traffic to flow both ways.

We can have three different actions for traffic in the policy-maps.

Pass – Traffic gets through but not return traffic is permitted. Useful for “stateless” protocols like RIP
Inspect – Allow traffic through and also allow the return traffic back.
Drop – Drop the traffic

If we have a policy-map that allows some traffic through, the rest of the traffic not matching any class will be implicitly dropped, this is even if we don’t specify a class class-default.

That are the most important things you need to be aware of when configuring this feature.

To enable AAA we need the AAA new-model command but what does it really do? Many of us makes assumptions about this command.

By default if we have an empty config then we will be able to use the console and get straight into enable mode (priv15). If we try to telnet in (VTY) then we can’t login since no password has been set. If we set a password then we can login to priv 1 but we won’t be able to enable since no enable password has been set.

When configuring AAA we use method lists. We can use the list called ‘default’ or create our own. The sneaky thing about aaa new-model is that when we enable this the ‘default’ list goes active which is applied to the VTY. What surprised me is that this is not applied to the console. Someone had a theory that Cisco wanted to apply it to both console and VTY but too many users got locked out of their routers so they had to back on this implementation, true or not, I don’t know.

When aaa new-model has been enabled the device will ask for local authentication. If we haven’t defined any users then no access for you (VTY-nazi). Console will still work though, we will have to enable to enter priv 15 as usual.
Now if we define a user we will be able to login remotely as well, we do need to configure an enable password to get into priv 15 though.

For the lab I have seen that if people get a task with AAA they will create a new method list with no authentication and no authorization and apply it to the console and VTY. As I pointed out we should not have to enable this to the console but better safe than sorry I guess. This can be configured in the following way:

aaa new-model
aaa authentication login VTY none
aaa authorization exec VTY non
line con 0
login authentication VTY
authorization exec VTY
line vty 0 4
line authentication VTY
authorization exec VTY

How would you configure this, what do you do in real life? Post in comments.

I’m going through the blueprint and now I checked off IP accounting. The feature is very simple, it lets us see which source destination pairs that are sending traffic to each other. We can also configure to look what precedence values that are in the packets. There is also an option to look at the MAC-addresses of the packets passing through and also packets that are being denied by an access-list. The topology is dead simple, see below.

Configure your routing protocol of choice to get reachability. I’m using OSPF, it does not matter at all as long as you have connectivity. Now lets say that we are interested in which source and destination pairs that are sending traffic THROUGH the router (transit). Packets destined TO the router will not be seen in the accounting. I’ll configure accounting on R2′s interface to R1 and then initiate a ping from R1 to R3. I’ll send traffic both to the loopback and R3′s FastEthernet interface to see two different source/destination pairs.

OK, lets ping.

Now we will check the accounting database with the show ip accounting command.

So that shows us what sources/destinations are sending traffic to each other, interesting! We can also see the number of packets and number of bytes. If we want to check statistics for only certain hosts we can use the global ip accounting-list command to define what hosts we are interested in. We define hosts/networks as in ACL with network/wilcard combination.

Storing entries in the IP accounting database requires some memory, there could be a risk of exhaustion if we have too many entries but the default is set to max 512 entries. We can define this with the global ip accounting-threshold command.

So now we want to check what IP precedence values pass through our interfaces and also what MAC addresses that are sending/receiving traffic. Lets configure this.

Then we send some pings from R1, I will send with a ToS of 128, what IP precedence/DSCP is that? Think quick.

Lets verify at R2 if we see anything, the command to use is show interface precedence.

So a ToS of 128 was a IP prec of 4 but you already figured that, right? What is that traffic with IP prec 6? Mysterious…We are running routing so that is OSPF which is marked with an IP precedence of 6 automatically by the router itself. We can also check what MAC addresses have been learned.

Here we also see OSPF represented by the MAC address 01-00-5E-00-00-05. We can also see when the last packet was sent which is quite handy. Now we will turn on accounting for access-lists as well, first we will define an ACL denying ICMP to 3.3.3.3 which is the loopback of R3. Note that we need the log keyword in the ACL.

Now we send traffic from R1 to 3.3.3.3.

For some reason I don’t see anything with the show ip accounting access-violations. Maybe this is a software issue? I tried turning off CEF as well. If any of my readers get this working I would be interested.

Lastly lets have a brief look at how traceroute works in IOS. Cisco devices uses UDP traceroute compared to ICMP used by Windows. The router sends packets with TTL of 1 and then N+1 the further away the probe goes. Traceroute sends three packets for every hop. The first hop will have a destination port of 33435, the second one will have 33436 and so on. If we want a router to not respond to traceroute we can turn off IP unreachables. Note that this will not hinder traceroute for which this router is not the final destination. Only the final device will send an ICMP unreachable (port unreachable) which is ICMP code 3. The other routers will send time exceeded which is ICMP code 11.

If we did want to block traceroute going through the router we could block this with an ACL denying packets that have ttl-exceeded or all packets lower than a certain TTL. If we need to find ICMP codes we can reference the ASA library. This should be available at the lab. You can find the reference by following this path.

Products > Security > Firewalls > Firewall Appliances > Cisco ASA 5500 Series Adaptive Security Appliances > Configure > Configuration Guides > Cisco ASA 5500 Series Configuration Guide using the CLI, 8.4 > Reference > Addresses, Protocols, and Ports > ICMP types

So this is just another feature that is handy to have.

Lab is coming up real fast now. I am checking off boxes in the blueprint for the stuff I feel comfortable with. I have maybe 70% checked but some of the stuff I want to revisit is QoS, multicast and a few filtering scenarios for routing and also NAT. There are also a few services that I want to check briefly in case I get them at the lab.

I don’t feel 100% prepared but I do feel that I do have a chance if everything goes well. All I can do is my best. I will also review a few labs down the final stretch. The last few days I will try to keep my brain fresh instead of cramming by only doing light review and by visiting the gym.

These topics are probably not very likely to appear at lab but it still good to at least have seen them once so you don’t get stumped if you should get a task like that at the lab.

Frame relay multilink (MFR) is defined in FRF.16.1 as defined by The Frame Relay Forum. See this URL for complete specification.

The basic idea is to take several frame-relay interfaces with the same DLCI and bundle them into one logical interface. Kind of like an Etherchannel. The reason I write this post is that the DOCCD isn’t really intuitive for this topic and there does not seem to be a lot of documentation how to configure this rather simple feature.

I will be using a simple topology where router R1 is dually connected to R2 and R3 is also dually connected to R2. R2 will be acting as the frame switch, we could have used a separate frame switch in Dynamips but this is a bit more fun and shows how to configure the SP side as well.

The configuration on the customer side is rather simple. First we create the logical interface which is named mfr and then a number. We will use number 1. All configuration like IP address and access-lists or anything like that goes to this interface.

Then we have to define which interfaces are part of the bundle. This is done with the encapsulation frame-relay mfr command.

Then we do the same thing on the other side but with a different IP address of course. Then we can verify that the link is up with show frame-relay multilink. We verify with a ping.

If you want to check that both links are being used then you can clear the counters and then do a ping. The number of packets should be equal or close to.

This is how a show frame pvc looks.

Note that the multilink interface is shown here instead of the physical interfaces. The MFR interface works the same way as a regular frame relay interface. Since I’m using a physical interface all DLCI’s will be mapped to it automatically and inverse ARP is used to resolve the remote layer 3 address to the local DLCI.

We also have the option of running the MFR interface on a subinterface, both as multipoint or point-to-point. Multipoint does not really make sense in this case but it can be done. The regular rules apply, if using multipoint we can either use a frame map statement or the frame-relay interface-dlci and rely on inverse ARP. For point-to-point interfaces we just use the frame-relay interface-dlci command as usual.

Now to the configuration of the FR switch. We enable frame-relay switching as usual. The configuration for the MFR is the same as for the customer side but we need to define that this interface is DCE and then we have the frame-relay route statements which map to the MFR interfaces instead of physical interfaces.

This is the configuration of R2.

Now we will look at MLPPPoFR which is another way of doing bundling of links by using PPP. First we start with the basic configuration. We bind the DLCI’s to the virtual template.

We do the same configuration on R2 and then we will configure the virtual-template to use multilink functionality.

You will see that several virtual-access have been created. We can verify with show ppp multilink command.

That is basically it. Now you know how to configure FRF.16 and MLPPPoFR.

So I’m going through the blueprint checking off everything before the lab. I know FR pretty well by now but there are always some details you forget. As I was going through FR again I thought about possible failure scenarios and restrictions that could be used at the lab. Here is a sample task I thought of.

Router R1 is running frame-relay on interface serial0/0. Via LMI 4 PVC’s have been learned, DLCI 102, 103, 104 and 105. Disable inverse ARP for all DCLI’s except 102. Do not use the no frame-relay inverse arp command. For this task you are allowed to create an additional interface.

Post solution in comments.

So lab is now only some weeks away. This is what I’m up against, the blueprint.

The final weeks I will try to go through this list and work on my weaker areas.

This post is about route redistribution and the different filtering techniques
we have available in our toolbelt. This post requires that you have a basic
understanding of route redistribution. For some good posts look at Petr
Lapukhovs posts at INE.

First lets define what is route redistribution? Generally we will use route
redistribution when multiple routing protocols are running in the network or
multiple instances of the same routing protocol is running. This can be due
to mergers, acquisitions or a fork lift upgrade. Maybe network is running
OSPF but is migrating to EIGRP or vice versa. We can also have redistribution
of connected and static routes.

What are some of the issues we can run into with route redistribution? We can
have routing loops in the network. These may not always be visible right away.
We can see them when doing a traceroute or when using debug ip routing.

We can also have issues with route feedback. Route feedback is when a redistributed
route gets redistributed back into the same protocol from which it originated. This
can lead to suboptimal routing or routing loops.

How do we define how “believable” a prefix is? First we must know that only the
same prefixes will be compared. 162.14.1.0/24 and 162.14.1.0/25 are not the same
prefixes. Longer match always wins! If we are comparing the two same prefixes
from different routing protocols then the AD will determine which one is the
best. Lower AD wins. If the AD for some reason is the same then the default AD
is the tie breaker.

Routes that are external should be less trusted than internal routes. EIGRP does
this by default by setting AD to 170 for external routes but 90 for internal. If
the other protocols did the same then we would not have issues with routing loops
at all. OSPF uses the same AD for internal and external prefixes (110) but we
can modify the AD for external routes if we want to. RIP does not have this
possibility.

Routing loops generally occur when we have redistribution between a protocol with
higher AD to a protocol with lower AD. This means that RIP is most often involved
in loops since it has the highest AD and we can’t define an external AD.

Lets look at a topology where loops can occur. This image is hand drawn and something
I do when doing labs to try to spot potential issues. I use a different color for
different protocols and draw arrows where redistribution occurs.

We are doing mutual redistribution on R1 and R2. The issue here is that we will
have suboptimal routing. You can download topology and configs here.

Look at the show ip route and traceroute from R1 to R2′s loopback.

R1 is going the whole way round even though it has a direct route via RIP to
R2. This is of course due to RIP having a higher AD than OSPF. Lets look at
a few different ways of fixing this. If this was the CCIE lab you would do nothing
unless it was asked of you to provide optimal routing. We don’t have a loop so it’s
not really a big issue at the moment. The issue here is that OSPF is not setting
a higher AD on it’s external prefixes. So we will have to do this manually.

So we set the AD to something higher than RIP, 121 in this case. Now we take
the direct path. Remember that AD is a local setting so this would have to be done
on all routers choosing suboptimal path.

We could also lower the AD of RIP. Either we do it for all routes or for a selection
of routes. Here we will select the routes with an ACL. We can set the distance for
all route sources or for a specific one based on the IP. Here we only have one so
we don’t really care, we will match on 0.0.0.0 255.255.255.255.

So now the AD is 109 for the RIP route which beats OSPF of 110. This would of course
also be have to be done on all routers with suboptimal path.

This is another way of doing it. We are setting a distance of 255 for the RIP routes
when they are entering as OSPF routes. 255 is not a valid distance for installing to
RIB so RIP routes will be preferred.

We can also use a distribute-list to control what routes get installed via OSPF.
Since OSPF is a link state protocl the LSA will of course propagate to other
routers.

As you can see the route is still present in the OSPF database but it does not
get installed into the RIB.

There is also a more fancy way of using distribute-lists. We can tie them to
a routing-protocol and define what is allowed to go out from that protocol
into the routing protocol that we are currently configuring. We will configure
R2 so that RIP routes are not allowed to be redistributed into OSPF. This will
kill any redundancy in the network.

So we go to the config mode of the routing protocol we are redistributing into.
Then we define with the distribute-list what is allowed to go out from other
protocols into the one we are now configuring. This is an effective way of
filtering when we have a lot of redistribution going on. In a small scenario
like this it does not make much sense but it’s very handy in large scenarios.

There is still one tool left and that is the route-map. The route-map is the
most flexible and scalable solution of all. We can choose what prefixes get
redistributed with an access-list or prefix-list. Lets try that first.

Here we matched prefixes with a prefix-list. The prefix-list has a deny for the
loopback of R2 and permits anything else. The route-map only has a permit statement,
deny in prefix-list and permit in route-map means that the prefix does not match
and moves to the next statement which is an implicit deny. The permit matches the
permit of the route-map and allows anything else.

This is an example of route feedback.

R5 is the only redistribution point. The issue here is that the routes that R3
learns from R1 and R2 via RIP will arrive at R5. R5 then redistributes into
OSPF. R3 will receive these LSA’s and find that this path is better due to a
lower AD. R3 will then install this route. R3 stops announcing that route via
RIP. R5 looses its route via RIP and can’t redistribute it into OSPF, so it
stops announcing it via OSPF. R3 installs the RIP route again and the fun
has just begun. Debug ip routing will show this procedure repeat over and over.

For our final tool we need a more complicated scenario to make full use of it.
First lets take a look at the topology. Download configs and topology here.

We have mutual redistribution on R3 and R5. The issue here is that we are
redistributing into RIP with a seed metric of 1. R3 sees R1′s loopback via
RIP with a metric of 2. R3 redistributes this information into OSPF. R5 learns
this information via OSPF and then redistributes into RIP with a metric of 1.
R3 now has two possible paths to R1 loopback. One with a metric of 1 and one
with a metric of two. Of course the lower metric wins. This means that R5
points towards R3 via R4 and R3 points to R5. Ladies and gentlemen, we have
a routing loop. There is definately a risk of loops when doing mutual redistribution.
When I redistribute something into RIP I usually set a quite high seed metric like 7.
This lowers the risk of loops because the RIP metric internally should be lower unless
it’s a very large network.

The probably best way to filter redistribution is to use route tagging. We set
a tag in a route-map and then base our filters on this. Sometimes it can be difficult
knowing where a route originated and from which protocol it came. If we set good tags
we can see both just by looking at the tag. I usually set a tag like 390, that means
that router 3 originated the route and it came from EIGRP. A tag of 4120 would mean
that it was a RIP route from R4 to begin with. Now lets try this technique as well.

We will set a tag of 3120 on R3 and also deny routes with a tag of 5110 on R3. This
is used to prevent R3 from taking OSPF routes from R5 received over RIP and then
redistributing them back into OSPF.

This is what the filtering looks like on R5.

Here we deny routes with a tag of 3120 and set our own tag of 5110.

Note that there is a risk that the loop still remains. R3 is announcing
routes via RIP natively to R5. We have a chicken and egg problem here.
This is the scenario before tagging. R3 redistributes RIP into OSPF. R5 receives
the routes and install them via OSPF since AD is lower than RIP. R5 then redistributes
these back into RIP. R3 sees the lower metric via R5 and installs this route in the
RIB. We now have a loop.

We start implementing tagging. R3 tags RIP routes going into OSPF
and denies any prefixes that R5 has already redistributed from OSPF to RIP.
R5 denies routes from R3 with a tag of 3120 from going back into RIP and sets
its own tag of 5110. There is a risk that R5 sees the best path via RIP to R3
and then announces an OSPF route which R3 installs. We need to make sure that
R5 sees the best path via OSPF to have a stable network. This can be done by
clearing routing table and shutting down links. To make sure this does not
happen we should also tag RIP routes going into OSPF on R5.

So you see that redistribution can be very complicated and there are a lot of
tools available. Try to check what routes are native to which routing protocol
and make sure that these are preferred in that domain. You can use distance
or other tools to make sure this happens.

In the real lab there could be hidden bombs that you need to detect to have a stable
topology. Probably you won’t be that restricted what you could do but there could
be some restrictions. So it’s good to have as many tools as possible. If all else
fails, do what it takes to get connectivity. Use distribute-lists or whatever
to have a stable topology. Yes, you will loose points but you can definately
not finish the lab if you don’t have a stable topology.

I hope this post has been informative and if you want to give med feedback
post in the comments section.

This is a follow up post to my last OSPF post about repairing area 0.
In the comments section Ray asked me what we can do if we have a
scenario where we have another area that is discontigous. In this
example we are using area 1. Area 1 is used everywhere but between
R1 and R4 is the backbone area. The routes will therefore be
interarea. What if we were told to make these routes appear as
intra area? First look at the topology. Download .net file and
initial and final configs here.

First we start by confirming that routes received on R1
are coming in as interarea routes.

Yes they are. This is expected behaviour. So what can we do to
make them appear as intraarea? Using a virtual link does not help
since it always belongs to area 0. We could use the tunnel technique
that was used in the previous post. Let’s try that. Same procedure as
last time. Source tunnel from physical interface. Use IP unnumbered
from an interface in area 1.

The tunnel is up and we have a new adjacency.

The IP address is 0.0.0.0/0 and located in area 1. Lets look at the
router LSA for area 1.

The address is 0.0.0.16 just as the last time. This should be 10 according
to the SNMP MIB but as one of my friends Patrick pointed out 16 in decimal
is 0×10 in hex. Maybe it is encoded in hex?

So now lets check the routing table.

Problem solved. Routes are now received as intraarea. So there you have it,
another OSPF problem solved.