Route Reflector Client Design Question

Mailman List Mirror (Read Only)
1

I have a RR Client design question......

CORE1-------------------2x10G-----------------------CORE2

                                                                                  >
                                                                                  >
                               10G Ring |
                                                                                  >
                                                                                  >

PE1----------PE2----------PE3----------PE4----------PE5

-Core1 & Core2 are RR Reflectors with full IPV4 Tables (ASR9K)
-MPLS LDP Enabled
-IGP is ISIS
-Each PE peers only with Core1 and Core2 as RR Clients with iBGP
-PE's are only receiving a default route from the Core Routers due to TCAM size of 20K (ASR920's\ME3800's)
-The ring does not have that much traffic on it <500m, so I do not want to use additional 10G ports on the Core's and is why I have it in a 10G U ring.
-Primary link to the cores is via the PE1 --- CORE1 Like......... For this discussion the link between PE5 to CORE2 is set up as a backup link.

The scenario is I have traffic between PE2 and PE3. Since the PE's are only receiving a default route from the Cores. Traffic is label switch from PE2 - PE1 - Core1 does a IP lookup at Ingress then label switches back to PE1-PE2-PE3. This ends up being 5 hops and doubling the traffic on the link to the Cores.

My questions is how do I get traffic to go directly between the PE's without going to the Core Routers?

1. Can I enable iBGP between the PE's in a full mesh to allow traffic between the PE's without going to the core's. Or does this break the Route Reflector model?
2. Create a route policy on the Core's advertising routes learned from the PE's back to all the PE's on the ring.
3. Is this one of the down sides to U Rings?
4. Leave it alone and move on to bigger and better things....

Thanks

Erik

2

Hey Erik,

1) This messes up the design and introduces unnecessary complexity. As your issue is not directly caused by having a RR topology, I’d avoid doing that.
2) That, IMHO, would be the optimal solution, assuming you don’t have enough internal routes to overflow the TCAM of your PEs in the near future. This would also solve some potential loops if you ever want to pass unlabelled traffic.
3) That’s a very generic question with broad potential answers.
4) You could, but you’d have to evaluate that trade-off yourself :wink:

Kind regards,

My thoughts and words are my own.

Spyros

    I have a RR Client design question......
    
    CORE1-------------------2x10G-----------------------CORE2
    > >
    > >
    > 10G Ring |
    > >
    > >
    PE1----------PE2----------PE3----------PE4----------PE5
    
    -Core1 & Core2 are RR Reflectors with full IPV4 Tables (ASR9K)
    -MPLS LDP Enabled
    -IGP is ISIS
    -Each PE peers only with Core1 and Core2 as RR Clients with iBGP
    -PE's are only receiving a default route from the Core Routers due to TCAM size of 20K (ASR920's\ME3800's)
    -The ring does not have that much traffic on it <500m, so I do not want to use additional 10G ports on the Core's and is why I have it in a 10G U ring.
    -Primary link to the cores is via the PE1 --- CORE1 Like......... For this discussion the link between PE5 to CORE2 is set up as a backup link.
    
    The scenario is I have traffic between PE2 and PE3. Since the PE's are only receiving a default route from the Cores. Traffic is label switch from PE2 - PE1 - Core1 does a IP lookup at Ingress then label switches back to PE1-PE2-PE3. This ends up being 5 hops and doubling the traffic on the link to the Cores.
    
    My questions is how do I get traffic to go directly between the PE's without going to the Core Routers?
    
    1. Can I enable iBGP between the PE's in a full mesh to allow traffic between the PE's without going to the core's. Or does this break the Route Reflector model?
    2. Create a route policy on the Core's advertising routes learned from the PE's back to all the PE's on the ring.
    3. Is this one of the down sides to U Rings?
    4. Leave it alone and move on to bigger and better things....
    
    Thanks
    
    Erik

3

I have a RR Client design question......

CORE1-------------------2x10G-----------------------CORE2
>
         >
>
         >
> 10G Ring
>
>
         >
>
         >
PE1----------PE2----------PE3----------PE4----------PE5

-Core1 & Core2 are RR Reflectors with full IPV4 Tables (ASR9K)
-MPLS LDP Enabled
-IGP is ISIS
-Each PE peers only with Core1 and Core2 as RR Clients with iBGP
-PE's are only receiving a default route from the Core Routers due to TCAM
size of 20K (ASR920's\ME3800's)
-The ring does not have that much traffic on it <500m, so I do not want to
use additional 10G ports on the Core's and is why I have it in a 10G U ring.
-Primary link to the cores is via the PE1 --- CORE1 Like......... For this
discussion the link between PE5 to CORE2 is set up as a backup link.

The scenario is I have traffic between PE2 and PE3. Since the PE's are
only receiving a default route from the Cores. Traffic is label switch from
PE2 - PE1 - Core1 does a IP lookup at Ingress then label switches back to
PE1-PE2-PE3. This ends up being 5 hops and doubling the traffic on the link
to the Cores.

My questions is how do I get traffic to go directly between the PE's
without going to the Core Routers?

1. Can I enable iBGP between the PE's in a full mesh to allow traffic
between the PE's without going to the core's. Or does this break the Route
Reflector model?

#1 is fine and works

4

Hello,

In order to accept only the default route, I assume that you want to have internet access to the ASR920 inside a vrf. ?!?
If this is the case, your consideration of the default route and the TCAM size is correct. But, if there is internet traffic between the PE2-PE3 in the same vrf , then I think that its ok to leak more specific prefixes from PE2 to PE3 (by using specific Route Targets) from the CORE1 & CORE2 (RR).

Unless there is something that I miss, option #2, is more favorable.

Michalis Bersimis

5

If I have understood your design correctly then don't use
next-hop-self on the RR's. Ideally you'd have out of band RRs so you
don't need NHS but I can see that you're are both RRs and PEs so NHS
is required. If you enabled iBGP sessions between all your PEs you
have defeated the point of RRs :slight_smile:

I don't know your setup in detail but is there something you can do on
your RRs to allow you to remove NHS? eBGP routes will have NHS by
default so you just need to check the impact on iBGP routes, maybe you
can remove NHS at the expense of redistributing a couple of connected
routes.

Cheers,
James.

6

Erik,

Before I email my suggestions, can you clarify the followings;

Do Core1 and Core2 also provide the function of BDRs peering with your
upstream/s?

Or

Just acting as Core/RRs with 500Mbps of traffic traversing through them?

Cheers
Ahad

7

Ahad,

The Cores have
Connections to other POPs
Transiting
IX circuits connected on them
Also have some Customer on them so they do also act like a PE.

Thanks

Erik

Erik Sundberg
Sr. Network Engineering
Network Engineering Department
p: 773.661.5532
c: 708.710.7419
e: esundberg@nitelusa.com
Main: 888.450.2100
NOC 24/7: 866.892.0915
350 North Orleans Street, Suite 1300N Chicago, IL 60654
www.nitelusa.com<http://www.nitelusa.com/>
[Nitel]
Managed Telecom Services
MPLS | Ethernet | Private Line | Internet | Voice | Security

8

My questions is how do I get traffic to go directly between the PE's without going to the Core Routers?

1. Can I enable iBGP between the PE's in a full mesh to allow traffic between the PE's without going to the core's. Or does this break the Route Reflector model?

You could do, but then you lose the point of the RR in the first place,
as it's likely your Metro-E nodes will continue to grow, making this
iBGP mesh thing, well, messy.

2. Create a route policy on the Core's advertising routes learned from the PE's back to all the PE's on the ring.

You could do, but adds unnecessary routing complexity since the role of
an RR is to, well, reflect.

3. Is this one of the down sides to U Rings?

Not really a downside, just the perks of extending IP/MPLS all the way
into the Access (I drink to the death of Layer 2 Metro-E networks - my
liver will probably give out before I do, though...).

4. Leave it alone and move on to bigger and better things....

Now where's the fun in that :-)?

So we've solved this problem by using BGP-SD (Selective Download):

  * For every prefix each Metro-E node handles, originate that toward
    both RR's with NEXT_HOP=self.

  * Attach a BGP community along with the routes originated toward the
    RR's. For maximum saving of your precious FIB in your Metro-E nodes,
    use a BGP community that is unique to the ring. This way, you don't
    need to accept routes into each Metro-E's FIB that don't require the
    "local" forwarding.

  * Ensure the RR's reflect the routes they learn from each Metro-E node
    to the other Metro-E nodes.

  * Setup BGP-SD on each Metro-E node. Match the ring-specific BGP
    community you added to each Metro-E node's prefix origination.
    Accept those routes into FIB + default. Reject everything else (from
    populating the FIB).

That should give you local forwarding within the ring, while maintaining
very sparse population of your Metro-E nodes' FIB's.

Mark.

9

Mark,

Your solutions sounds like the best one.

We have just started to mess with Selective download and we have only turned it on for one of the PE’s in our network. I am in the process of upgrading our Core routers from Cisco12410 to ASR9906’s, before I turn this one. Having the PE decide what to import is a better solution than trying to do router filtering on the core routers.

Thanks for the info

Erik

10

Mark Tinka wrote:

11

So our good friends at Cisco, after promising that they'd have code for
IOS XE/CSR1000v by this time 2018, told me that implementation for it on
this code base has moved to 2H'19, no guarantees. I'm too bored with
them to even be irritated.

Someone was telling me that the IOS XR implementation only supports the
main IPv4/IPv6 address families, so their 6PE network cannot enjoy their
ORR deployment yet.

Mark.

12

I'm not sure what you are taking about with ORR, but I use dual RR's for a redundant cluster with me ASR9k's in IOS XR, and I have them handling routes for ...

Family l2vpn VPLS
Family vpnv4
Family vpnv6

...so my 6PE mpls l3vpn has been working fine

Aaron

13

https://tools.ietf.org/html/draft-ietf-idr-bgp-optimal-route-reflection-16

Mark.