Increase bandwidth usage in partial-mesh network?

Looking for recommendtions or suggestions…

I’ve got a downstream customer asking for help; they have a private internal network that I’ve taken to calling the “partial-mesh network from hell”: it’s got two partially-overlapping radio networks, mixed with islands of isolated fiber connectivity.
Dynamic routing protocols (IS-IS, OSPF, EIGRP, etc.) generally will only select the best path, they won’t spread the load unless all paths are equal - and they are very unequal in this network, ECMP would likely fail horribly.

The network is becoming bandwidth-limited, so they’re wanting to make use of all available paths, not just the single “best” path. It’s also remote and spread out, so adding new links or upgrading existing links is difficult and expensive.
Oh, and their routers are overdue for a refresh, so acquiring replacement h/w is now possible.

Has anyone come across any product or technology that can handle the multi-path-ness and the private-network-ness like a regular router, but also provides the intelligent per-flow path steering based on e.g. latency, like an SD-WAN device (and/or some firewalls)?

Here’s hoping,

-Adam

Hey! From the description it must be one of our clients!

Just beware if you go this route, a network that is probably already unstable and unreliable will become at least an order of magnitude worse. You can’t fix ten lbs of stuff into a 4 lb stuff bag. The internet protocols do not tolerate congestion well.

It sounds like they need to get back to the basics first.

Simplification, in lieu of added complexity, seems to be the appealing approach.

Mark.

The babel protocol does some of this.
https://datatracker.ietf.org/doc/html/rfc6126

Regards,
Bill Herrin

Hah, no not your client :slightly_smiling_face:. Their existing network is actually surprisingly stable, but it is bandwidth-constrained. As well as the various other replies I’ve seen here and off-list (THANKS!), the only commercial product I’ve found so far that might have a hope of handling this is HPE/Aruba’s Silverpeak line. We’ll see what else comes out of the woodwork, though - if nothing else, it’s a very interesting exercise!

Assuming that the reasons for the low bandwidth and use of radio is due to physical constraints - distances, inhospitable terrain between nodes, etc. In this case, some good 'ol MPLS traffic engineering using LSP’s with bandwidth reservations may be the way to influence how traffic is routed. Then, they may need some platform to provide observability and potentially dynamic re-routing of LSP’s based on actual or predicted congestion situations. If traffic patterns and utilization are not ideally deterministic, then skip the bandwidth reservation and ensure that the automation is in place to reroute traffic when necessary.

I know, adding complexity, but if you just can’t build the links you would want, this may be a way to work with what you’ve got.

I would agree.
Not sure if other vendors have something similar, but in Juniper land you could use traffic engineering with container lsp to go a step further than just plain rsvp-te.

Kind regards
Karsten



Von: nanog@nanog.org
Gesendet: 14. Oktober 2021 03:06
An: athompson@merlin.mb.ca
Antworten: mrodriguez@fletnet.com
Cc: nanog@nanog.org
Betreff: Re: Increase bandwidth usage in partial-mesh network?

Assuming that the reasons for the low bandwidth and use of radio is due to physical constraints - distances, inhospitable terrain between nodes, etc. In this case, some good 'ol MPLS traffic engineering using LSP’s with bandwidth reservations may be the way to influence how traffic is routed. Then, they may need some platform to provide observability and potentially dynamic re-routing of LSP’s based on actual or predicted congestion situations. If traffic patterns and utilization are not ideally deterministic, then skip the bandwidth reservation and ensure that the automation is in place to reroute traffic when necessary.

I know, adding complexity, but if you just can’t build the links you would want, this may be a way to work with what you’ve got.

Maybe add a little bit of linear optimization on top of faucet/openvswitch/openflow to calculate best paths based upon bandwidth, paths, and fill-factors. There is a presentation where Google uses that technique to obtain high utilization on their links (not necessarily those tools though).

Raymond Burkholder

Has anyone come across any product or technology that can handle the multi-path-ness and the private-network-ness like a regular router, but also provides the intelligent per-flow path steering based on e.g. latency, like an SD-WAN device (and/or some firewalls)?

Maybe add a little bit of linear optimization on top of faucet/openvswitch/openflow to calculate best paths based upon bandwidth, paths, and fill-factors. There is a presentation where Google uses that technique to obtain high utilization on their links (not necessarily those tools though).

Raymond Burkholder

This is what a large Italian wisp has done, here are a couple of presentations made at our ITNOG sessions.
I’m not sure if they have open sourced anything though.

Brian

Maybe something like this (if you can break it into different bgp ASNs by network area):

“draft-mohanty-bess-ebgp-dmz-03” https://datatracker.ietf.org/doc/html/draft-mohanty-bess-ebgp-dmz-03