Large Scale BGP Route Reflection

Lesson Learned:

Larger Scale BGP desisgn cannot be serviced by only a single RR

--Single RR is a single point of failure

RR Clusters allow redundancy and Hierarchy

--Cluster is defined by the clients a RR servers

--RRs in the same Cluster use the same Cluster-ID

Inter-Cluster peerings between RRs can client of non-client peerings

--Depends on redundancy design.

Using multiple RR’s in cluster, for example RR1, RR2, and RR3. Each with client connected to each RR.

The ultimate design goal is when a client for RR1 sends an update within the cluster to the RR. These updates are passed down to the clients in the Cluster. The update is then passed to the other clusters then in turn sends the updates down to their clients.

Then based on the rules of RR’s, this will determine whether or not the route is updated. Example, if RR1, RR2, and RR# are all clients of each other, if they are clients, the updates will be passed along, if not then the updates will fail.

Note: there is technically nothing wrong with RR’s being clients of each other. The only issue is there will be more overhead in the Control Plane. Then even though the route is looping between the RR’s, the Cluster list is going to prevent that loop from occurring. In a large scale design you could be processing 3 or 400,000 routes.

The Case where you would want the RRs to be clients of each other, is if there was a link failure between the RR’s and if the RR’s are non-clients, it would affect the updates between all RRs.

TOPOLOGY:

From the Topology The Configs for RR1 and it’s clients will look like this:

RR Config:

R1:

Router bgp 100

neighbor R1CLIENTs peer-group

neighbor R1CLIENTs remote-as 100

neighbor R1CLIENTs route-reflector-client

neighbor R1CLIENTs update-source Loopback1

neighbor 2.2.2.2 peer-group R1CLIENTs

neighbor 3.3.3.3 peer-group R1CLIENTs

---------

CLIENT Config:

router bgp 100

neighbor 1.1.1.1 remote-as 100

neighbor 1.1.1.1 update-source loopback1

Cluster 2:

RR Config:

R4:

Router bgp 100

neighbor R4CLIENTs peer-group

neighbor R4CLIENTs remote-as 100

neighbor R4CLIENTs route-reflector-client

neighbor R4CLIENTs update-source Loopback1

neighbor 5.5.5.5 peer-group R4CLIENTs

neighbor 6.6.6.6 peer-group R4CLIENTs

CLIENT Config:

router bgp 100

neighbor 4.4.4.4 remote-as 100

neighbor 4.4.4.4 update-source loopback1

Cluster 3:

RR:

Router bgp 100

neighbor R7CLIENTs peer-group

neighbor R7CLIENTs remote-as 100

neighbor R7CLIENTs route-reflector-client

neighbor R7CLIENTs update-source Loopback1

neighbor 8.8.8.8 peer-group R7CLIENTs

neighbor 9.9.9.9 peer-group R7CLIENTs

CLIENT Config:

router bgp 100

neighbor 1.1.1.1 remote-as 100

neighbor 4.4.4.4 update-source loopback1

Once these are all up we will need to verify the peerings.

We can do a #sh ip bgp sum – we should see on all the RR – the peers for all the neighbors:

Ex:

R1#sh ip bgp summary

BGP router identifier 1.1.1.1, local AS number 100

BGP table version is 1, main routing table version 1

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd

2.2.2.2         4   100      37      37        1    0    0 00:34:50        0

3.3.3.3         4   100      37      37        1    0    0 00:34:53        0

4.4.4.4         4   100       6       6        1    0    0 00:02:50        0

7.7.7.7         4   100       6       6        1    0    0 00:02:52        0

R1#

Also – from RR4 –

R4#sh ip bgp

BGP table version is 20, local router ID is 4.4.4.4

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,

              r RIB-failure, S Stale

Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path

* i1.1.1.0/24       1.1.1.1                  0    100      0 i

*> 4.4.4.0/24       0.0.0.0                  0         32768 i

r>i7.7.7.0/24       7.7.7.7                  0    100      0 i

R4#

We can see the routes and how they’re learned.

interface Loopback10

 ip address 100.100.100.9 255.255.255.0

end

So now on R9 – lets add a route and advertise it and see how BGP updates the route.

R9(config)#router bgp 100

R9(config-router)#network 100.100.100.0 mask 255.255.255.0

We can see from another RR in the topology that the route has been updated from one RR to another:

*>i100.100.100.0/24 9.9.9.9                  0    100      0 i

R7#

Note:  If there’s a case where there’s more than 1 RR per cluster, first we would need to specify the cluster ID be the same on both RRs – to keep loops form happening. Also we might want to set the local preference because the Local Preference is for the entire AS.

For example:

On R1 – I’ll set the local preference to be 100 and on R4 I’ll set the preference to be 200.

R4:

R4#sh route-map

route-map PREF_200, permit, sequence 10

  Match clauses:

  Set clauses:

    local-preference 200

  Policy routing matches: 0 packets, 0 bytes

Then under the BGP process

I’ll tell neighbor 7.7.7.7 (R7) to use this weight

R4(config-router)#neighbor 7.7.7.7 route-map PREF_200 in

Note: this change will not apply until we ask for a route refresh. For this lab I’ll just clear ip bgp.

   Network          Next Hop            Metric LocPrf Weight Path

*>i1.1.1.0/24       1.1.1.1                  0    100      0 i

*> 4.4.4.0/24       0.0.0.0                  0         32768 i

r>i7.7.7.0/24       7.7.7.7                  0    200      0 i

*>i100.100.100.0/24 9.9.9.9                  0    200      0 i

Now we can see the routes form R7 have a preference of 200.

Note: The convergence of the BGP network, is more a function if the underlying IGP that BGP itself. Because we’re always relying on the IGP route to the next hop, which means if we can get high-availability for the next hop, it doesn’t really matter what the BGP update is.