Hello all. I am having a hard time trying to articulate why a Dual Home ISP should have full tables. My understanding has always been that full tables when dual homed allow much more control. Especially in helping to prevent Async routes.
Am I crazy?
It’s pretty pointless for a small ISP to get full routes, because the BGP tables are so highly manipulated. It’s better to just get “company” routes for each upstream, and then use your own traffic engineering via prepending and static or policy routes to balance the outbound traffic the way you like.
-mel
Honestly, this. Your only real choice is what of 2 pipes to chuck it out of.
Full tables vs partial and a default don’t make the process much more intelligent for 1 site dual homed, and as mentioned routing policy will have more influence.
-Ben
We have full tables from 2 ISPs at just one datacenter, and it is nice in the case of partial reachability issues—If one ISP loses access to routes to a destination but the other one doesn’t, for example. For us, the decision to do full tables was easy, as we are running 2 MX150s which can very easily handle the load and convergence is still less than a minute or so. As far as optimal path goes, full tables really doesn't help us much, so we made sure to get matching speed circuits just to make things simple. We have AT&T and CenturyLink, and most things prefer CenturyLink as they are pretty well peered due to all of their acquisitions. It would be interesting to see a distribution plot of ASPATH length, I would bet that a huge chunk of our routes are only 2-3 hops away.
/chris
Honestly, this. Your only real choice is what of 2 pipes to chuck it out of.
Full tables vs partial and a default don’t make the process much more intelligent for 1 site dual homed, and as mentioned routing policy will have more influence.
-Ben
Dear Brian,
Hello all. I am having a hard time trying to articulate why a Dual Home ISP should have full tables. My understanding has always been that full tables when dual homed allow much more control. Especially in helping to prevent Async routes.
The advantage of receiving full routing tables from both providers is that in cases where one of the two providers is not yet fully converged, your routers will use the other provider for those missing destinations. This may happen during maintenance or router boot-up in your upstream’s network.
Another advantage of receiving full routes is that you can manipulate LOCAL_PREF per destination, or compose routing policy based on per-route attributes such as BGP communities your upstreams set. It can happen that a provider is great for 99% of destinations, except a few - without full tables such granular traffic-engineering can be cumbersome.
Virtually all internet routing is asymmetric, I wouldn’t consider that an issue.
Am I crazy?
I dropped out of university, never completed my psychology studies, I fear I am unqualified to answer this question. 
Kind regards,
Job
fre. 24. jan. 2020 18.23 skrev Job Snijders <job@instituut.net>:
Am I crazy?
I dropped out of university, never completed my psychology studies, I fear I am unqualified to answer this question.
Education shopping, it is called by some.
Chriztoffer
Hello all. I am having a hard time trying to articulate why a Dual Home ISP should have full tables. My understanding has always been that full tables when dual homed allow much more control. Especially in helping to prevent Async routes.
If you're multi-homed you'll have some asymmetric routes. Even if you aren't, they'll be asymmetric to your upstream. It's easy to manipulate which path traffic leaving your site takes, difficult or impossible to manipulate via which path it enters.
If you have sufficient horsepower and memory in your routers, taking full tables won't hurt anything and will help in situations where one of your upstreams loses reachability to a destination. For belt-and-suspenders, take full routes plus a default.
Am I crazy?
Perhaps, but if so it has little if any relationship to the size of the BGP tables in your border routers.
Don’t forget to connect to peering exchanges and take their full routes too if you can.
Full tables will not make much noticeable difference if you are not peering. However you want to make sure both links get used. It can be a 90%/10% split but 100%/0% is bad because then you may discover that the alternate path is actually broken the moment the primary fail. If you choose only default then you need to think about that.
If you join any peering exchanges, full tables will be mandatory. Some parties will export prefixes and then expect a more specific prefix received from your transit to override a part of the space received via the peering.
Regards
Baldur
fre. 24. jan. 2020 17.41 skrev Brian <brian.bsi@gmail.com>:
Brian, you're correct that having more routing information will give you more control over routing decisions. However, for me, it's not about control/traffic engineering but about basic redundancy/availability. Taking full tables gives you visibility into whether an actual path exists (or does not exist) via each of your upstream providers. If you just take a default route originated by your peer you'll only have visibility that there's a connection between your router and your peer - not to the rest of the internet or to a specific destination past your peer. As Job said, maintenance, router reboots, and other upstream connectivity issues can cause an upstream peer to be missing routes that the other peer might have.
In my experience, most folks multi-home for improved redundancy/availability. So for those that are multi-homed, I recommend taking full tables so that you can get the most benefit. This may come at an additional expense so I understand why some would choose to take a default route only.
--Blake
90/10 will suck when the link carrying 90% of your traffic needs more pipe and you have a ton of unused capacity on the other one. Full tables from both providers gives you more options to tune things (assuming outbound is your larger direction). If you're an eyeball provider and most of your traffic is inbound, your outbound traffic routing decisions aren't quite as relevant.
Have those suggesting "multihoming with two partial feeds and default routes" forgotten peering pissing matches, long lasting inter-network capacity issues, or that certain "tier 1" providers don't even have/provide a full v6 table?
If you're going to multihome, do it right, and get full routes from all your providers.
If you don't have full tables you will certainly have a default route somewhere, either set statically by you or advertised by your upstream.
Accidents may happen: your ISP may blackhole a destination; sometimes they adjust loads, sometimes their redundancy is not properly set up or they may have an otherwise incorrect BGP configuration. Sometimes they just mess up.
If you have full tables, you will simply stop getting the advertisements for the bad destinations from the bad ISP and your routers will take care of it because they will keep getting the advertisements from the other exit.
If you don't have full tables and the mistake is in a destination for which the default route is used, your traffic will most probably be lost because they don't have enough information to know a different exit and you may get a call at midnight. It may or may not happened to me. 
Octavio.
lør. 25. jan. 2020 00.40 skrev Jon Lewis <jlewis@lewis.org>:
Full tables will not make much noticeable difference if you are not peering. However you want to make sure both
links get used. It can be a 90%/10% split but 100%/0% is bad because then you may discover that the alternate path
is actually broken the moment the primary fail. If you choose only default then you need to think about that.
If you join any peering exchanges, full tables will be mandatory. Some parties will export prefixes and then expect
a more specific prefix received from your transit to override a part of the space received via the peering.90/10 will suck when the link carrying 90% of your traffic needs more pipe
and you have a ton of unused capacity on the other one. Full tables from
both providers gives you more options to tune things (assuming outbound is
your larger direction). If you’re an eyeball provider and most of your
traffic is inbound, your outbound traffic routing decisions aren’t quite
as relevant.
If your goal is to maximize your capacity, you should run a default route with equal cost multi path for perfect load balancing. Just beware that there is effectively no redundancy when exceeding the capacity of a single link.
Also consider the typical two transits each connected to a separate router, each router handling a single circuit. I will wager that the majority of such dual homed organisations have no idea that those two routers by default will make different routing decisions. You get more control but you also need the experience and talent to use it. For many it might be better to have a solution that is understood.
Have those suggesting “multihoming with two partial feeds and default
routes” forgotten peering pissing matches, long lasting inter-network
capacity issues, or that certain “tier 1” providers don’t even
have/provide a full v6 table?
The solution is to stay clear of tier 1 networks. Find a good local tier 3. Whatever you are going to do, they will do better.
Regards
Baldur
* Baldur Norddahl
If you join any peering exchanges, full tables will be mandatory. Some parties will export prefixes and then expect a more specific prefix received from your transit to override a part of the space received via the peering.
That would be a fundamentally flawed expectation, in my opinion.
An AS that that advertises a prefix to its peers must be prepared to carry traffic to that entire prefix via that peering circuit.
There is simply no guarantee that a more-specific prefix advertised somewhere else will make it into the RIBs and FIBs of all the peers of that AS.
The AS might of course opt to do so anyway for traffic engineering purposes, but there is no assurance that it will actually work 100% of the time. When it doesn't, the AS in question would need to carry the traffic from the peering circuit across their own backbone.
If the AS in question for some reason cannot do so, it would need to adjust its advertisements across the peering circuit so as to avoid falsely advertising reachability to unreachable destinations.
Tore
lør. 25. jan. 2020 13.42 skrev Tore Anderson <tore@fud.no>:
- Baldur Norddahl
If you join any peering exchanges, full tables will be mandatory. Some parties will export prefixes and then expect a more specific prefix received from your transit to override a part of the space received via the peering.
That would be a fundamentally flawed expectation, in my opinion.
I do not disagree, however the real world sometimes works differently. Like anycast, people break the rules and gets away with it.
In any case, this is from a recent personal experience. We had a problem that led us to drop full tables and run with a default for a while. Nobody noticed the difference, which is why I can confidently say that unless you need the full tables for something, the advantages are somewhat overstated.
However one customer found a reachability problem. Turns out that a peer was exporting a /19 prefix through a peering session with us and at another site they exported a /24 from the same space with no routing between sites.
Regards
Baldur
I’m listening to the advice of others and taking it in….
For my ISP, I’ve had 2 or 3 internet uplinks for about 12 years now for 50,000 subs, and have only learned a default route on them. It’s been good up to this point.
-Aaron
Interesting discussion.
Besides the point about control which many people have made, I would like to point - it also depends on your geography and peering in that geography. Cannot generalise but typically in the US and Europe you will find reasonably good peering across larger operators and hence cases of your traffic taking really odd path (like going out of country and in forward / reverse direction) would be low. That may not be the case in certain other geographies like say Asia or Africa. In my home country India I find occasional cases of traffic leaking out of the country.
I recently posted this trace taken from an Indian RIPE Atlas probes to a popular content site with frontend mostly on Akamai: https://atlas.ripe.net/measurements/23857778/#!probes
Traceroute to www.hotstar.com (104.123.148.87), 48 byte packets
1 172.16.0.1 0.943ms 0.828ms 0.821ms
2 172.16.1.100 0.743ms * 13.019ms
3 103.145.6.4 AS139540 0.684ms 3.548ms 2.972ms
4 103.145.6.3 AS139540 5.035ms 17.172ms 0.664ms
5 59.162.52.161 59.162.52.161.static.vsnl.net.in AS4755 0.92ms 0.845ms 1.218ms
6 115.112.167.241 115.112.167.241.STATIC-Kolkata.vsnl.net.in AS4755 4.522ms 21.518ms 29.118ms
7 172.25.87.174 13.919ms 13.741ms 13.509ms
8 172.31.180.57 53.947ms 93.642ms 58.971ms
9 180.87.36.9 ix-ae-4-2.tcore1.cxr-chennai.as6453.net AS6453 54.4ms 53.752ms 53.721ms
10 180.87.36.41 if-ae-34-2.tcore1.svq-singapore.as6453.net AS6453 85.424ms 85.835ms 86.098ms
11 120.29.215.39 AS6453 86.247ms 85.818ms 86.203ms
12 49.45.4.87 AS64049 84.675ms 84.555ms 84.614ms
13 49.45.4.87 AS64049 83.941ms 83.835ms 84.163ms
14 103.198.140.185 AS64049 85.437ms 85.498ms 85.506ms
15 172.16.23.9 100.334ms 100.817ms 172.25.106.39 100.344ms
16 172.16.2.70 98.977ms 98.835ms 98.795ms
17 172.16.2.41 99.69ms 99.628ms 99.565ms
18 172.16.0.210 102.005ms 102.059ms 101.929ms
19 104.123.148.87 a104-123-148-87.deploy.static.akamaitechnologies.com AS55836 105.386ms 105.358ms 105.284ms
Ignoring the DNS based mapping and associated challenges of Akamai and speaking purely from routing. Say if AS139540 had full table it would simply have picked direct route to AS55836 which is one of it’s upstream (https://bgp.he.net/AS139540#_graph4).
Thus 100ms would be down to probably sub 20ms levels. Whether or not such issues affect you depends on whether upstreams or upstream’s upstream peer in the region or not. In this case upstream AS4755 & AS55836 are not exchanging traffic locally. Furthermore their upstream AS6453 and AS64049 are also not exchanging traffic locally either. So if I have to decide, I will always pick full table in these cases. AS_PATH will help and one can always tweak for larger known cases.
Dear Job and NANOG,
Just wondering, wouldn’t any of you guys consider using full tables in this case, for the ability to detect and avoid prefix hijacks (using RPKI/ROV or other means)?
Of course, I’m focused on security, and I know this is often not a high priority for a real network manager who has many other considerations; just want to know. Thanks.
So all our transit comes from the top 7 "global" carriers. Yes,
including Cogent :-).
But that only accounts for about 15% of our overall traffic. The rest
comes from peering.
Mark.
I'd really like to hear more datapoints from different eyeballs on this, like
60% local caches, of remaining traffic, 70% peered
so transit = 0.4*0.3 = 12%
I think this might be reasonable, but perhaps it's even less transit
for eyeballs today?