Monday, May 19, 2014

Administrative Distance Based Routing Loops, Debug IP Routing, IP Route Profile

Lessons Learned:

Identifying Routing Loops.
Visually trace the route advertisement path
--ideally you should be able to predict 100% of routing loops before they occur
CLI tools
-Connectivity testing with ICMP via TCL
-debug ip routing
-ip route profile
-Traceroute

------------------------------------------------------- 

If the route is looping, even ping’s and traceroute’s might not tell where the source of the problem is. This where the ip route profile and debug ip route are good for.

TOPOLOGY:
BB3 will peer RIP with R4 – R4 will peer EIGRP with R5 and R4 will peer  OSPF with R6.



















On R4 – lets redistribute RIP in to EIGRP.

R4(config)#router eigrp 1
R4(config-router)#redistribute rip metric 100000 100 255 1 1500

Verify the routes we expect are showing up correctly:
D EX    10.10.10.0 [170/2221056] via 172.16.5.5, 00:00:52, FastEthernet0/0
D EX 11.0.0.0/8 [170/2221056] via 172.16.5.5, 00:00:52, FastEthernet0/0
D EX 12.0.0.0/8 [170/2221056] via 172.16.5.5, 00:00:54, FastEthernet0/0
D EX 13.0.0.0/8 [170/2221056] via 172.16.5.5, 00:00:54, FastEthernet0/0

Now let’s redistribute EIGRP in RIP
R4(config)#router rip
R4(config-router)#redistribute eigrp 1 metric 1

Verify the networks are as expected on the BB# router:
R    172.16.0.0/16 [120/1] via 10.10.10.4, 00:00:11, FastEthernet0/0

Now we should verify the traffic path, currently router 4 is the only router doing redistribution so we should only traverse that path.

BB3#traceroute 172.16.23.2
Type escape sequence to abort.
Tracing the route to 172.16.23.2

  1 10.10.10.4 16 msec 40 msec 20 msec
  2 172.16.45.5 16 msec 20 msec 24 msec
  3 172.16.5.10 40 msec 16 msec 44 msec
  4 172.16.10.2 48 msec *  56 msec
BB3#

Now on R# we will do redistribution from OSPF and EIGRP


R3(config)#router ospf 1
R3(config-router)#redistribute eigrp 1 subnets

R3(config)#router eigrp 1
R3(config-router)#redistribute ospf 1 metric 100000 100 255 1 1500

Now let’s verify the routes – we will at the routing table of R10.
R10#traceroute 192.168.69.6

Type escape sequence to abort.
Tracing the route to 192.168.69.6
  1 172.16.10.2 16 msec 20 msec 28 msec
  2 172.16.23.3 24 msec 48 msec 36 msec
  3 192.168.93.9 48 msec 40 msec 56 msec
  4 192.168.69.6 68 msec *  84 msec
R10#

Once all the redistribution is complete – we can check to see if routes are being added or removed and collect statistic on the routing table to see if the network is stable or f there are changes in the routing table.
The “ip route profile” feature is designed to accomplish. We will need to configure this on all the routers.

EX: R10(config)#ip route profile

Now that the feature is turned on – we can look at the output by simply saying
R3#sh ip route profile

IP routing table change statistics:
Frequency of changes in a 5 second sampling interval
-------------------------------------------------------------
Change/   Fwd-path  Prefix   Nexthop  Pathcount  Prefix
interval  change    add      change   change     refresh
-------------------------------------------------------------
0         5         5        6        6          6        
1         0         0        0        0          0        
2         0         0        0        0          0        
3         1         1        0        0          0        
4         0         0        0        0          0        
5         0         0        0        0          0        
10        0         0        0        0          0        
15        0         0        0        0          0        
20        0         0        0        0          0        
25        0         0        0        0          0        
30        0         0        0        0          0        
55        0         0        0        0          0        
80        0         0        0        0          0        
105       0         0        0        0          0        
130       0         0        0        0          0        
155       0         0        0        0          0        
280       0         0        0        0          0        
405       0         0        0        0          0        
------------------------------------------------------------
Change/   Fwd-path  Prefix   Nexthop  Pathcount  Prefix
interval  change    add      change   change     refresh
-------------------------------------------------------------
530       0         0        0        0          0        
655       0         0        0        0          0        
780       0         0        0        0          0        
1405      0         0        0        0          0        
2030      0         0        0        0          0        
2655      0         0        0        0          0        
3280      0         0        0        0          0        
3905      0         0        0        0          0         
7030      0         0        0        0          0        
10155     0         0        0        0          0        
13280     0         0        0        0          0        
Overflow  0         0        0        0          0        
R3#

This is basically taking a sample every 5 seconds, so did the number of routes go up, down, etc. Did the next hop change, did the flooding change?

We can read this like this:
The first row says - There were 0 changes over a 5 second interval. The number of intervals of which that occurred we’re 5 intervals. Basically there we’re 0 changes in 5 seconds 5 times so far.
If we see the changes increase over time especially in the high intervals – 3 – 80 for ex. That’s bad, means there’s an issue:

-------------------------------------------------------------
Change/   Fwd-path  Prefix   Nexthop  Pathcount  Prefix
interval  change    add      change   change     refresh
-------------------------------------------------------------
0         67        67       81       81         81       
1         0         0        0        0          0        
2         0         0        0        0          0        
3         14        14       0        0          0        
4         0         0        0        0          0        
5         0         0        0        0          0        
10        0         0        0        0          0        
15        0         0        0        0          0        
20        0         0        0        0          0        
25        0         0        0        0          0        
30        0         0        0        0          0        
55        0         0        0        0          0        
80        0         0        0        0          0        
105       0         0        0        0          0        
130       0         0        0        0          0        
155       0         0        0        0          0        
280       0         0        0        0          0        
405       0         0        0        0          0  

In general the bolded values should all be zero and the values at the top should all be counting up…
There are some issues here, so 14 times there was 3 changes over a 5 (samples are in 5 seconds) second interval there issues with routers begin added and Fwd-path changing…
3         14        14       0        0          0        

This will show routing table stability and instability. This will not show what is the exact issues are, the Key is the feature will help diagnose issues.
For the most part this will show either convergence of the network OR some type of flapping topology going on in the network.
The route profile will work better for AD based loops than Metric based loops.

To determine the actual issues – we should most likely turn-on debug ip routing
Note: in large scale networks it might be a better idea to send the log outputs to the buffer instead of the console.

R4#debug ip routing
IP routing debugging is on


*Mar  1 00:20:48.739: RT: add 11.0.0.0/8 via 10.10.10.3, rip metric [120/1]
*Mar  1 00:20:48.739: RT: NET-RED 11.0.0.0/8
*Mar  1 00:20:48.747: RT: SET_LAST_RDB for 12.0.0.0/8
  NEW rdb: via 10.10.10.3

The first part of the output says we received 11.0.0.0/8 via 10.10.10.3 (BB3) via RI with a distance of 120 and a metric of one.
This is now is installed in the routing table.


*Mar  1 00:20:48.747: RT: add 12.0.0.0/8 via 10.10.10.3, rip metric [120/1]
*Mar  1 00:20:48.751: RT: NET-RED 12.0.0.0/8
*Mar  1 00:20:48.755: RT: SET_LAST_RDB for 13.0.0.0/8
  NEW rdb: via 10.10.10.3

*Mar  1 00:20:48.759: RT: add 13.0.0.0/8 via 10.10.10.3, rip metric [120/1]
*Mar  1 00:20:48.763: RT: NET-RED 13.0.0.0/8
*Mar  1 00:20:48.995: RT: closer admin distance for 11.0.0.0, flushing 1 routes
*Mar  1 00:20:48.999: RT: NET-RED 11.0.0.0/8
*Mar  1 00:20:49.003: RT: SET_LAST_RDB for 11.0.0.0/8
  NEW rdb: via 192.168.46.6

*Mar  1 00:20:49.007: RT: add 11.0.0.0/8 via 192.168.46.6, ospf metric [110/20]
*Mar  1 00:20:49.011: RT: NET-RED 11.0.0.0/8
*Mar  1 00:20:49.047: RT: closer admin distance for 12.0.0.0, flushing 1 routes
*Mar  1 00:20:49.047: RT: NET-RED 12.0.0.0/8
*Mar  1 00:20:49.055: RT: SET_LAST_RDB for 12.0.0.0/8
  NEW rdb: via 192.168.46.6

This Now says for prefix 11.0.0.0/8 – there’s now a “closer admin distance” closer admin distance via OSPF. With an AD of 110 and metric of 20.
This will now override the RIP Metric.
I’ve omitted the reset of the output abut it showed that over and over the routed was added to the table via RIP – then quickly removed and added via OSPF.

Another way to test is run an ICMP ping with a timeout of 1 – we should see intermittent connectivity.
We could also ten turn on “debug ip icmp” to see who is exactly sending us the ICMP unreachable messages.


I now need to turn on “debug ip routing”on all the other routers.
On R5 -

*Mar  1 01:19:16.123: RT: no routes to 11.0.0.0
*Mar  1 01:19:16.127: RT: NET-RED 11.0.0.0/8
*Mar  1 01:19:16.131: RT: delete network route to 11.0.0.0
*Mar  1 01:19:16.131: RT: NET-RED 11.0.0.0/8
*Mar  1 01:19:16.163: RT: delete route to 13.0.0.0 via 172.16.45.4, eigrp metric [170/2195456]
*Mar  1 01:19:16.167: RT: SET_LAST_RDB for 13.0.0.0/8
  OLD rdb: via 172.16.45.4, Serial0/0

So this is telling us that the R4 router is removing the route to into EIGRP.


If I go to another router in the topology – R3

*Mar  1 01:21:30.635: RT: no routes to 11.0.0.0
*Mar  1 01:21:30.635: RT: NET-RED 11.0.0.0/8
*Mar  1 01:21:30.635: RT: delete network route to 11.0.0.0
*Mar  1 01:21:30.635: RT: NET-RED 11.0.0.0/8
*Mar  1 01:21:31.663: Periodic IP routing statistics collection
R3#
R3#

This also say’s I had a router to 11.0.0.0 then it was redrawn. This would lead us back to our router doing the redistribution (R4).

*Mar  1 01:24:20.799: RT: add 11.0.0.0/8 via 192.168.46.6, ospf metric [110/20]
*Mar  1 01:24:20.799: RT: NET-RED 11.0.0.0/8
*Mar  1 01:24:20.827: RT: closer admin distance for 12.0.0.0, flushing 1 routes
*Mar  1 01:24:20.827: RT: NET-RED 12.0.0.0/8
*Mar  1 01:24:20.831: RT: SET_LAST_RDB for 12.0.0.0/8
  NEW rdb: via 192.168.46.6

Once again we see that we’re deleting the RIP route and installing the OSPF route…
One the RIP route is deleted, it means that it cannot be redistributed into EIGRP and then it cannot be redistributed into OSPF.
So the withdrawn will happen over one and over. We now know the issues is related to the Administrative Distance.

So if we changed the distance so the RIP route is lower is it will it should correct the issue. The problem is that filtering in redistribution is not going to help.
One way to corrected this is to just have R4 perform the redistribution itself into both protocols. This will keep R3 (currently redist between EIGRP and OSPF) from learning the routed from EIGRP because of the AD.
The EIGRP routed will not get installed into the route table, so they cannot be redistributed.

So on R4 – if we do the redistribution –

R4(config)#router ospf 1
R4(config-router)#redistribute rip subnets

R4(config)#router eigrp 1
R4(config-router)#redistribute ospf 1 metric 1 1 1 1 1

R4(config)#router rip
R4(config-router)#redistribute ospf 1 metric 1

R3#sh ip route 11.0.0.0
Routing entry for 11.0.0.0/8
  Known via "ospf 1", distance 110, metric 20, type extern 2, forward metric 3
  Redistributing via eigrp 1
  Advertised by eigrp 1 metric 100000 100 255 1 1500
  Last update from 192.168.93.9 on FastEthernet1/0, 00:01:00 ago
  Routing Descriptor Blocks:
  * 192.168.93.9, from 4.4.4.4, 00:01:00 ago, via FastEthernet1/0
      Route metric is 20, traffic share count is 1

We can now see the router is installed via OSPF and not EIGRP.

The only problem with this – is that if one of the links goes down and the routes are then learned from another protocol we would have a routing loop again.
Basically an order of operations again because only what’s then in the routing table will be able to be redistributed.

To correct this issue completely we need to tell the redistributing router what routes to use what protocols.


Monday, May 12, 2014

Routing Loops Overview, EIGRP Route Loop Prevention.

Lessons Learned:

Why Routing loops Occur:
Redistribution causes loss of end-to-end route calculation and loop prevention
--EX: DUAL not compatible with SPF
Looping cannot occur with single redistribution point
--Internal routing process prevents this
Loops can occur if multiple redistribution points and wrong path is selected due to –
---Wrong path has lower metric that correct path
---Wrong path has lower AD than correct path

------------------------------------------------------------- 

Identifying Routing Loops
Visually trace the route advertisement path
--Ideally we should be able to predict 100% of the loops before they occur

CLI tools
-Connectivity testing with ICMP via TCL
-debug ip routing – Will show what protocol the routes are being learned from, what is the metric value, AD value and whether the route is being added or deleted over and over. IF the topology is not changing – this is less likely to be a redist issue.
-ip route profile –Used to collect statistics on the routing over a period of time.
-traceroute –if packets are bouncing back and forth – will be an issue.

Normally the issue with redistribution – is that if there is a loop in the topology, it’s time consuming to find the problem and then resolution.

Preventing Route Feedback.
How to prevent looping depends on why loop occurred
Metric based loops
-Fixed with ACL, Prefix-List, or tad filtering and AD
Administrative Distance based loops
-Generally fixed only with AD

------------------------------------------ 
There are two main categories of Loops. Loops that are based on the Metric and loops that are based on the distance. The reason we need to separate the two of them is there are fixed that we can apply to Metric based loops that will not work on AD based loops.

Metric based loops are the easiest to prevent by filtering the routes at the redistribution points. We can apply a Tag to the route or manually specific ACL and prefix list on what routes we don’t want to redist.

With AD based loops – filtering at the redistribution point will generally not accomplish anything. 
There’s a difference when a routing is choosing two routes inside the same protocol and looking at the metric value vs looking at two routes form different protocols and looking at the one with the lower AD.

Whether the loops are Metric or AD based – the key is to look at the network diagram and follow the route advertisement path on a hop-by-hop basis, to predict for a particular router. What interface is the route going to come in on, and if it is learning from multiple interfaces and the same protocol which are we going to choose based on the metric. Also if it’s coming in through multiple interfaces through multiple protocols which is it going to choose based on the lower administrative distance.

TOPOLOGY:
--------------------


















If you do redistribution across a single router – there’s really little change if any you can cause a routing loop. When there are multiple routers preforming redistribution between protocols that’s when you run the risk of routing loops.
  
If we have two routes redistributed from two routing protocols with the same longest match – we’re going to choose the one with the lower AD.

In the topology I have a currently have a single router preforming redistributing between RIP and OSPF.
---------------------------------
router eigrp 1
 network 172.16.41.4 0.0.0.0
 network 172.16.42.4 0.0.0.0
 no auto-summary
!
router ospf 1
 log-adjacency-changes
 redistribute rip subnets
 network 10.1.43.4 0.0.0.0 area 0
 network 42.42.42.4 0.0.0.0 area 0
!
router rip
 version 2
 redistribute ospf 1 metric 1
 network 192.168.47.0
 no auto-summary---------------------------------- 

Now lets redistribute EIGRP into RIP and EIGRP into OSPF. And RIP and OSPF into EIGRP.
R4(config)#router rip
R4(config-router)#redistribute eigrp 1 metric 1

R4(config)#router ospf 1
R4(config-router)#redistribute eigrp 1 subnets

R4(config)#router eigrp 1
R4(config-router)#default-metric 100000 100 255 1 1500
R4(config-router)#redistribute ospf 1
R4(config-router)#redistribute rip

Note: here I specified the default metric of 100000 100 255 1 1500

100000 – This is the bandwidth = to 100Mbp
100 –  delay in tens of microseconds, if we say 100 that’s equal to 1000 microseconds
255 – this is the reliability = 100% reliable
1 –  load – a load of 1 means the minimum load.
1500 – this the MTU – really the only time the MTU will come into play as a tiebreaker if you have multiple equal routes. Most of the time the effective MTU will be 1500 or less.

Now once we redistribute the RIP and OSPF we should see them both getting the same metric.

We can verify local EIGRP topology and see if the RIP and OSPF Routes are getting redistributed.  And they are….

R4#sh ip eigrp topology

P 10.1.16.0/24, 1 successors, FD is 51200  --OSPF Route
        via Redistributed (51200/0)
P 10.1.43.0/24, 1 successors, FD is 51200 --OSPF Route
        via Redistributed (51200/0)
P 42.42.42.0/24, 1 successors, FD is 51200
        via Redistributed (51200/0)
P 10.1.36.0/24, 1 successors, FD is 51200 --OSPF Route
        via Redistributed (51200/0)
P 192.168.47.0/24, 1 successors, FD is 51200 –RIP Route
        via Redistributed (51200/0)
P 172.16.41.0/24, 1 successors, FD is 28160 –EIGRP LOCAL ROUTE
        via Connected, FastEthernet0/1
P 172.16.42.0/24, 1 successors, FD is 2169856
        via Connected, Serial0/0
P 172.16.25.0/24, 1 successors, FD is 33280
        via 172.16.41.1 (33280/30720), FastEthernet0/1
        via 172.16.42.2 (2172416/28160), Serial0/0
P 172.16.15.0/24, 1 successors, FD is 30720
        via 172.16.41.1 (30720/28160), FastEthernet0/1
R4#

As long as the Metric value is specifics, doesn’t matter what as long as it’s there and we have the correct process numbers then the redistribution should be processed locally.

There’s some cases where the routing process gets hung. This can normally be corrected by clearing the routing table .

EX: R4#clear ip route *
This will repopulate the routing and CEF tables.

So now what is the end result of the redistribution?
If we go to R6 for example – we should see the RIP networks, EIGRP and local OSPF Networks.

R6#sh ip route

O E2 192.168.47.0/24 [110/20] via 10.1.36.3, 00:37:29, FastEthernet0/0 –RIP Network
     172.16.0.0/24 is subnetted, 4 subnets
O E2    172.16.41.0 [110/20] via 10.1.36.3, 00:21:04, FastEthernet0/0 –EIGRP Networks
O E2    172.16.42.0 [110/20] via 10.1.36.3, 00:21:04, FastEthernet0/0
O E2    172.16.25.0 [110/20] via 10.1.36.3, 00:21:04, FastEthernet0/0
O E2    172.16.15.0 [110/20] via 10.1.36.3, 00:21:04, FastEthernet0/0 –EIGRP Networks
     42.0.0.0/24 is subnetted, 1 subnets
O       42.42.42.0 [110/66] via 10.1.36.3, 00:37:29, FastEthernet0/0
     10.0.0.0/24 is subnetted, 3 subnets
C       10.1.16.0 is directly connected, FastEthernet0/1
O       10.1.43.0 [110/2] via 10.1.36.3, 00:37:31, FastEthernet0/0
C       10.1.36.0 is directly connected, FastEthernet0/0
R6#

---------------------------------
If we want more visibility as to where the routes are coming form we can go to redistribution router (R4) and change the Metric type. Example we can set the Metric type for the redistributed routes to type 1 – so they show as E1 Routes on R6 now.

EX:
R4(config)#router ospf 1
R4(config-router)#redistribute eigrp 1 metric-type ?
  1  Set OSPF External Type 1 metrics
  2  Set OSPF External Type 2 metrics

R4(config-router)#redistribute eigrp 1 metric-type 1
% Only classful networks will be redistributed
R4(config-router)#

Note the error message is because I did not include the subnets keyword.

R4(config-router)#redistribute eigrp 1 metric-type 1 subnets
R4(config-router)#

------------------------------------------------ 
R6#sh ip route

O E2 192.168.47.0/24 [110/20] via 10.1.36.3, 00:44:05, FastEthernet0/0
     172.16.0.0/24 is subnetted, 4 subnets
O E1    172.16.41.0 [110/22] via 10.1.36.3, 00:02:09, FastEthernet0/0
O E1    172.16.42.0 [110/22] via 10.1.36.3, 00:02:09, FastEthernet0/0
O E1    172.16.25.0 [110/22] via 10.1.36.3, 00:02:09, FastEthernet0/0
O E1    172.16.15.0 [110/22] via 10.1.36.3, 00:02:09, FastEthernet0/0
     42.0.0.0/24 is subnetted, 1 subnets
O       42.42.42.0 [110/66] via 10.1.36.3, 00:44:05, FastEthernet0/0
     10.0.0.0/24 is subnetted, 3 subnets
C       10.1.16.0 is directly connected, FastEthernet0/1
O       10.1.43.0 [110/2] via 10.1.36.3, 00:44:06, FastEthernet0/0
C       10.1.36.0 is directly connected, FastEthernet0/0
R6#

Now we have a clearer picture of where the routes came from. The route type so far is not going to matter, we’re currently only using one router as the redistribution point- R4.

The one issue I do have is that R2 is learning routes form BOTH EIGRP and OSPF. Also all the routes in the routing table are local and not redistributed or external routes.

Routers in the OSPF Domain can get to local routes on R2 but not routes behind R2.
The only external route I have is to the RIP Domain, that is learned from OSPF.
R2#sh ip route

O E2 192.168.47.0/24 [110/20] via 42.42.42.4, 01:03:58, Serial0/1
     172.16.0.0/24 is subnetted, 4 subnets
D       172.16.41.0 [90/33280] via 172.16.25.5, 01:04:52, FastEthernet0/0
C       172.16.42.0 is directly connected, Serial0/0
C       172.16.25.0 is directly connected, FastEthernet0/0
D       172.16.15.0 [90/30720] via 172.16.25.5, 01:04:51, FastEthernet0/0
     42.0.0.0/24 is subnetted, 1 subnets
C       42.42.42.0 is directly connected, Serial0/1
     10.0.0.0/24 is subnetted, 3 subnets
O       10.1.16.0 [110/67] via 42.42.42.4, 01:04:00, Serial0/1
O       10.1.43.0 [110/65] via 42.42.42.4, 01:04:00, Serial0/1
O       10.1.36.0 [110/66] via 42.42.42.4, 01:04:00, Serial0/1

Also if we look at the EIGRP topology table we can see that there are routes in the topology table but they do not get installed into the routing table because they do not meet the feasibility condition. I this case – the problem is because OSPF has a lower administrative distance.

R2#sh ip eigrp topology
P 10.1.16.0/24, 0 successors, FD is Inaccessible
        via 172.16.42.4 (2195456/51200), Serial0/0
P 10.1.43.0/24, 0 successors, FD is Inaccessible
        via 172.16.42.4 (2195456/51200), Serial0/0
P 42.42.42.0/24, 0 successors, FD is Inaccessible
        via 172.16.42.4 (2195456/51200), Serial0/0
P 10.1.36.0/24, 0 successors, FD is Inaccessible
        via 172.16.42.4 (2195456/51200), Serial0/0
P 192.168.47.0/24, 0 successors, FD is Inaccessible
        via 172.16.42.4 (2195456/51200), Serial0/0
P 172.16.41.0/24, 1 successors, FD is 33280
        via 172.16.25.5 (33280/30720), FastEthernet0/0
        via 172.16.42.4 (2172416/28160), Serial0/0
P 172.16.42.0/24, 1 successors, FD is 2169856
        via Connected, Serial0/0
P 172.16.25.0/24, 1 successors, FD is 28160
        via Connected, FastEthernet0/0
P 172.16.15.0/24, 1 successors, FD is 30720
        via 172.16.25.5 (30720/28160), FastEthernet0/0
R2#

If we look at the actual route:
We can see the bandwidth is T1 – the total delay is 21000,
It shows the originating router is R4 (192.168.47.4).
Notice the Metric value should be 2195456 but the FD is 4294967295  which is the infinite value.

So when we see the FD at this value and the actual metric that has been calculated through the neighbor, this means we’re learning the route from some other protocol that’s stopping the EIGRP route from being installed. So because were not learning the route from EIGRP we cannot advertise the route into EIGRP

R2#sh ip eigrp topology 10.1.16.0/24
IP-EIGRP (AS 1): Topology entry for 10.1.16.0/24
  State is Passive, Query origin flag is 1, 0 Successor(s), FD is 4294967295
  Routing Descriptor Blocks:
  172.16.42.4 (Serial0/0), from 172.16.42.4, Send flag is 0x0
      Composite metric is (2195456/51200), Route is External
      Vector metric:
        Minimum bandwidth is 1544 Kbit
        Total delay is 21000 microseconds
        Reliability is 255/255
        Load is 1/255
        Minimum MTU is 1500
        Hop count is 1
      External data:
        Originating router is 192.168.47.4 
        AS number of route is 1
        External protocol is OSPF, external metric is 3
        Administrator tag is 0 (0x00000000)
R2#

Notice – When we s hip route for the prefix. The source protocol is not EIGRP.

R2#sh ip route 10.1.16.0
Routing entry for 10.1.16.0/24
  Known via "ospf 1", distance 110, metric 67, type intra area
  Last update from 42.42.42.4 on Serial0/1, 01:17:08 ago
  Routing Descriptor Blocks:
  * 42.42.42.4, from 172.16.41.1, 01:17:08 ago, via Serial0/1
      Route metric is 67, traffic share count is 1

R2#

So the issue seems to be that R2 is receiving the OSPF route and the EIGRP route. The AD of the OSPF route - 110 - is lower than 170 form EIGRP. So the EIGRP route is not getting installed in the routing table.  

To Fix this we can use would need to change the AD on R2 to prefer the EIGRP Routes.
Note: The AD value is only locally significant. So any changes will only affect R2.
Also note the with EIGRP we CANNOT change the external AD on a per prefix basis. OSPF, RIP, or BGP can be change on both internal and external routes. This only works on EIGRP for internal routes.

So let’s lower the AD on R2 from 170 to anything lower than 110.
From the routing process –

R2(config)#router eigrp 1
R2(config-router)#distance eigrp 90 109  -This basically says, keep the internal distance at 90 and change the external EIGRP to 109 (one lower than ospf)

Log output:
*Mar  1 01:49:54.391: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 1: Neighbor 172.16.25.5 (FastEthernet0/0) is down: route configuration changed
*Mar  1 01:49:54.403: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 1: Neighbor 172.16.42.4 (Serial0/0) is down: route configuration changed
R2(config-router)#
*Mar  1 01:49:58.355: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 1: Neighbor 172.16.25.5 (FastEthernet0/0) is up: new adjacency
*Mar  1 01:49:58.487: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 1: Neighbor 172.16.42.4 (Serial0/0) is up: new adjacency
R2(config-router)#

I will now once again clear the routing table:

We can now see that the external EIGP routes are getting installed with a distance of 109.
R2#sh ip route

D EX 192.168.47.0/24 [109/2195456] via 172.16.42.4, 00:00:07, Serial0/0
     172.16.0.0/24 is subnetted, 4 subnets
D       172.16.41.0 [90/33280] via 172.16.25.5, 00:00:07, FastEthernet0/0
C       172.16.42.0 is directly connected, Serial0/0
C       172.16.25.0 is directly connected, FastEthernet0/0
D       172.16.15.0 [90/30720] via 172.16.25.5, 00:00:07, FastEthernet0/0
     42.0.0.0/24 is subnetted, 1 subnets
C       42.42.42.0 is directly connected, Serial0/1
     10.0.0.0/24 is subnetted, 3 subnets
D EX    10.1.16.0 [109/2195456] via 172.16.42.4, 00:00:08, Serial0/0
D EX    10.1.43.0 [109/2195456] via 172.16.42.4, 00:00:08, Serial0/0
D EX    10.1.36.0 [109/2195456] via 172.16.42.4, 00:00:08, Serial0/0
R2#

We should now be able to advertise the routes in EIGRP because they’re now in the routing table.


Saturday, May 10, 2014

OSPF External Path Selection, TCL PING Scripting

Lessons Learned:

Redistribution from the RIP process into the OSPF process
When the RIP router originates routes as long as the OSPF area is a normal area these are going to be TYPE 5 LSAs in the OSPF database.  Either the E1 or E2 routes, By default the routes will show as E2 and the default metric is 20.

Under the OSPF process –
We simply say –

R4(config-router)#redistribute rip
% Only classful networks will be redistributed (this is a backwards compatibility feature – only clasful summarize will be included in the OSPF database)
R4(config-router)#

This will advertise the RIP routes with the default values.  To remove the classful subnets we need to add the keyword “subnets to the redist – process.

R4(config-router)#redistribute rip subnets
This should advertise anything running RIP including the connected interfaces running RIP into the OSPF database- TYPE 5 LSA”s

                Type-5 AS External Link States

Link ID         ADV Router      Age         Seq#       Checksum Tag
192.168.47.0    192.168.47.4    192         0x80000001 0x00C0AA 0

Or we can view the External
  LS age: 231
  Options: (No TOS-capability, DC)
  LS Type: AS External Link
  Link State ID: 192.168.47.0 (External Network Number )
  Advertising Router: 192.168.47.4
  LS Seq Number: 80000001
  Checksum: 0xC0AA
  Length: 36
  Network Mask: /24
        Metric Type: 2 (Larger than any link state path)
        TOS: 0
        Metric: 20
        Forward Address: 0.0.0.0
        External Route Tag: 0

All other OSPF routers in the topology should show the RIP routes as E2 routes:

R3#sh ip route
O E2 192.168.47.0/24 [110/20] via 10.1.43.4, 00:04:24, FastEthernet0/0

Also if we show the RIP route specifically

R3#sh ip route 192.168.47.0
Routing entry for 192.168.47.0/24
  Known via "ospf 1", distance 110, metric 20, type extern 2, forward metric 1
  Last update from 10.1.43.4 on FastEthernet0/0, 00:06:14 ago
  Routing Descriptor Blocks:
  * 10.1.43.4, from 192.168.47.4, 00:06:14 ago, via FastEthernet0/0
      Route metric is 20, traffic share count is 1

This says – the metric is 20 the type is 2 and the forward is 1 – the forward is to get to the ASBR.
This is a function of the E2 routes. They separate the seed metric plus the metric to the ASBR.
Note: the farther away from the ASBR the Forward metric will increment. If there is ever a tie in the E2 routes, we will look at the forward Metric for the best path.

Note: if we set these routes as External type 1 routes, then there’s no difference between the metric and the forward metric. The only time this really matter is when there are multiple exit points for the domain.

We can set the metric type during redistribution.
Verify –
We need to verify can we reach the destination from the other protocol.
R3#ping 192.168.47.7

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.47.7, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 16/32/44 ms
R3#

And I can reach my RIP router. If we can reach the destination we can assume the redistribution is correct. If the Routing domain is large there would be a lot of checks.
For Verification of a large domain we can use TCL scripts

TCL – stands for the Tool Command Language – this is an open standards language.
IOS – supports TCL 8.3.4
-99% of programming is outside the CCIS scope
-useful in redistribution for automating PING scripts

Only problem with TCL Scripts is the syntax has to be exact and there’s not a real reference guide

TCL PING Script Example:
R3#tclsh  (staring a TCL Shell)
R3(tcl)#foreach X {  (this says create a forloop and define a variable X, for each value that X has, run the ping command for the variables value. )

Ex:
tclsh
#foreach x {            
+>(tcl)# 10.1.43.4
+>(tcl)# 192.168.47.7
+>(tcl)# 172.16.41.1
+>(tcl)# 10.1.16.6
+>(tcl)#} { ping $X }


The end result is it will ping these IP’s in order. 

Friday, May 9, 2014

Route Redistribution Overview


Lessons Learned:

----------------------------------------------------

What routes are redistributed

How connected redistribution works

How IOS chooses routes to use

Why Routing loops can occur

How to identify routing loops

How to prevent routing loops

----------------------------------------------------

Route Redistribution –

Redistribution occurs from the routing table not the routing database.

When redistributing protocol X into Y, take.

-Routes in the routing table via protocol X

-Connected interfaces running protocol X

Route advertisement rules

-RIP vs EIGRP Vs OSPF VS BGP
 
Note the key point to understand is that the process of route redistribution occurs from the routing table and not the database.

Example – what’s installed in the EIGRP topology Vs what actually goes into the routing table.

When going between two protocols – taking source protocol X and putting it into Protocol Y, The router will first look at two things.

Ex: what are the routes actually installed in the routing table via that protocol and what are the connected interfaces, in the routing table that are running that protocol.
The first – in the case of EIGRP – what are the dynamic routes coming from our connected peers. Internal or external routes.

The connected interfaces – this will be based on the network statement that is running under the process.

So if we look at the “show IP protocols”, this will show us what are the connected links that care candidate to be redistributed between the processes.
 
Depending on the protocol – there are rules on what particular routes we can install in the table. And what routes we can advertise.

Connected Redistribution:
Implicitly occurs for connected links running the redistributed protocol

Additionally connected links can explicitly be include or excluded
-# redistribute connected (metric ) route-map.
-overrides implicit redistribution

When redistribution happens,. The router will be looking for two things, any dynamic routes in the routing table via that protocol. Then implicitly any interfaces that are running that process.

If we do not use the redistribute connected command, any interfaces in the source protocol that are directly connected will be redistributes into the destination protocol.

Note: there could be an issue – if we do explicitly list what links will be redistributed from the connected process – this will override what the implicit redistribution is.

We need to treat the dynamic routing process and the connected routing process as two different entities form the perspective of redistribution. As long as we do not manually issue the redistributed command – we should then assume that any interfaces running that protocol will be sent into the destination protocol.
 

How IOS chooses Paths:
Routing database choses one or more candidate paths.

-EIGRP via DUAL, OSPF via SPF, etc

-Load balancing via Maximum-paths.

If multiple equal matches between protocols…

-choose the lower administrative Distance

Install results in RIB and/or FIB.

Router will look through the database and find the actual paths that can go into the routing table. Then the router will look are there multiple matches for the exact prefixes from the different protocols.

|Ex: if we have a prefix 1.2.3.0 /24

And we’re learning it via both EIGRP and OSPF, then we will have to choose which one actually goes into the routing table. This will be based on the AD value. The lower AD will be candidate to go into the

RIB (routing table) and the FIB (the CEF table)

AD Values:

0 = Connected
115 = IS-IS
1 = Static
120 = RIP
5 – Eigrp Summary
160 = OD
90 = EIGRP internal
170 = Ext EIGRP
110 = OSPF
200 = Internal BGP
 
255 Infinite

 
Ex: if we’re learning and External OSPF route and an External EIGRP route, and they’re equal longest matches for the same destination

We would prefer to install the OSPF route and not the External EIGRP route.

This could be an issue because the advertising rules for EIGRP and RIP are different than OSPF and BGP. The Distance vector protocols can only advertise routes that get installed into the routing table.

OSPF and BGP do not have this limitation because with OSPF, all routers in the flooding domain have to have the same copy of the database. BGP – the candidate routes are the ones that will the BGP best path selection process.

RIP Redistribution – V2
Doesn’t differentiate between internal and external routes.

-Administrative distance of 120 for all routes

No default seed metric

-#redistribute (protocol ) metric (hops)

-#default-metric (hops)

There’s cases where RIP does not choose the correct path because the router cannot distinguish between internal and external routes.

There’s no way to separate in the routing process an internal RIP distance between an External RIP distance.

When we redistribute into RIP, OSPF or EIGRP into RIP – there is NO default seed metric or see hop count.

This means that we need to either specify globally under the process with the default Metric command or specifically for that protocol.

Example: RIP
# redistribute OSPF 1 Metric 5
# redistribute EIGRP 1 Metric 3

Either globally or individual protocol – if we do not specify the metric the routes will not be populated into the RIP database

Can verify this with the

#sh RIP database – this will tell us locally on the router that is doing the redistribution whether the process is actually occurring properly.

So if we Redistributed EIGRP into RIP – and we do not see the EIGRP routes as redistributed, then we know there could be an issue with the Metric or the routes are being filtered.

EIGRP Redistribution –
AD of 170 for external EIGRP
-Helps to automatically prevent route feedback.

Used Router-ID for loop prevention

No default seed metric unless EIGRP to EIGRP

#redistribute (protocol) metric (bandwidth) (delay) (load) (reliability) (MTU)

-#default-metric (bandwidth) (delay) (load) (reliability) (MTU)  

EIGRP – does differentiate between internal or External routes.
90 = Internal
170 = External

The idea behind this is if we have an external EIGRP route -

And some other IGP learned route for the same match.

If we learn the route through external EIGRP- this will be the least preferred path.  Because of the AD.

Note: EIGRP uses the router-id value for loop prevention of external prefixes. If our EIGRP Router-ID is 1.2.3.4 – when we redistribute OSPF into EIGRP – the EIGRP External prefixes are going to be tagged with the router-ID of 1.2.3.4 

This then means if the routing update leaves locally and goes out to some of the peers, then comes back in from another EIGRP neighbor – we’re going to discard the route. Any time you learn an external EIGRP route that has your route-id as the originating router the process is automatically going to deny this.

Just like RIP – EIGRP does not have a default seed metric unless we’re going through two spate EIGRP processes. EIGRP AS 1 and EIGRP 2 on same router – the individual composite values will be exchanged on a prefix by prefix basis, Load delay, mtu, etc – will be exchange

However if were coming from another protocol like OSPF into EIGRP.
We do need to manually specify what the seed metric value is.

We can use the redistribution under the process with the Redist protocol or set the default-metric.

There is also cases where if there is only a single point of redistribution – the seed metric value is not going to matter. Because if we have only one physical path form the RIP to EIGRP – it really doesn’t matter what the seed metric is. The Seed metric is really only going to matter when there is multiple points of redistribution and we’re doing traffic engineering or route filtering.


OSPF Redistribution -

AD of 110 for all OSPF routes

Uses router-id for flooding loop prevention

Default seed metric 20 and metric-type E2 / N2

OSPF path selection preference
-E1 > E2 > N1 > N2
-E1 & N1 vs E2 & N2 metrics

Does distinguish between external and internal, all will be assigned an AD of 110. This means OSPF will prefer the AD of 110 over the External EIGRP or RIP AD’s.

Under the routing process on the command line we can manually specify the AD for the External vs internal routes if we want to.

By default OSPF will choose the intra-area routes over the Inter-area routes and over E1 > E2 > N1 > N2

So regardless of metric value, etc – if we have an E1 route VS and N1 (NSSA) route we will always have to choose the E1 route

In OSPF the only time the redist route is going to matter is if we have multiple external matches of the same type prefix – then we use the metric. Ex two E1 routes of two E2 routes.

Does have default see of 20 – and default metric of E2 into a normal area and N2 into an NSSArea. These can be manually changed.

OSPF will use the route-id to prevent routing loops. Although OSPF is not a distance vector protocol.

BGP Redistribution –

Uses ORIGIN code Incomplete.

Normal EBGP and iBGP have no loop prevention

IGP to BGP

-Denies OSPF external routes by default

--- redistribute ospf (Pid) match internal external

BGP to IGP
-EBGP routes allowed, iBGP routes denied by default

-#bgp redistribute – internal

-legacy sysnc rule

-can cause a routing loop


BGP does Distinguishes between internal and external routes in two ways.

First is going to be based on the origin code that is set to incomplete or ? when you look at the #sh ip bgp.

BDP best path selection will prefer routes internal to the network – the ones that are configured under the network statement in the BGP process over the redistributed routes. Origin code for internal is a lower case “i

If all routes being equal – we will look at the origin code – the lower will be preferred IGP over incomplete.

For loop prevention we will use the same rules for normal ibgp and ebgp -

EBGP – uses the AS Path information – ex: if my AS is 100 and I receive a route in that has 100 in the path, then discard the update.

iBGP routes – anytime I learn a prefix from an iBGP neighbor, we will simply not advertise that prefix to another iBGP neighbor – unless I am a route reflector.

iGP to BGP – this is the normal way to advertise our internal prefixes to the global BGP network.  Ex: if we have a bunch of OSPF routes that we want to advertise into BGP, we can either use the network statement under the process  or just redistribute OSPF into BGP

The only exception for this by default the BGP process is going to deny the external OSPF routes. The logic is that if the route is external OSPF – it means that it came from some other protocol to begin with.

This only applies to OSPF – if we’re redistributing EIGRP – it will allow both the Internal or External routes to go.


Now going from BGP to IGP – in order to prevent loops inside our own internal ibgp network, we’re only going to allow external routes from BGP to OSPF or BGP to EIGRP.